How you do *unescape* a string... and some weirdness

[jyavenard]

Hello

In my qmake file, I added to a variable the following:

MYSQLIBS = "$$system(mysql_config --libs)"
MYSQLIBS ~= s/-arch +[a-z0-9_]+//g #remove all "-arch arg" settings
LIBS += MYSQLIBS

Unfortunately here, LIBS in the generated Makefile now contains:
LIBS = $(SUBLIBS) -L/usr/local/mythtv-trunk/lib -Wl,-Bsymbolic-functions\ -rdynamic\ -L/usr/lib/mysql\ -lmysqlclient


see that all the space have been escaped.

What I would like to end-up with is:
LIBS = $(SUBLIBS) -L/usr/local/mythtv-trunk/lib -Wl,-Bsymbolic-functions -rdynamic -L/usr/lib/mysql -lmysqlclient

Otherwise the following would fail to compile under linux.

Now that brings me to another weirdness.
The reason I did:

MYSQLIBS = "$$system(mysql_config --libs)"
instead of:
MYSQLIBS = $$system(mysql_config --libs)

It's because otherwise, the regex replacement that follow would have otherwise no effect at all, and it only works if the variable has been placed in quote.

Another annoyance I do not know how to deal with is: on mac, some linker settings are in the form:
-arch arg
where arg can be i386, x86_64, ppc7400, ppc64

The problem is how qmake deals when adding those parameters.
If I have

EXTRA_LFLAGS = -arch i386
and
QMAKE_LFLAGS = -arch i386

doing:

QMAKE_LFLAGS += $$EXTRA_LFLAGS

The generated makefile will contain:
LFLAGS = -arch i386 -arch

the 2nd i386 has been dropped. Compilation *will* failt
According to the qmake documentation, += shouldn't drop items, that's what *= does:
The *= operator adds a value to the list of values in a variable, but only if it is not already present. This prevents values from being included many times in a variable. For example:

Now I could do:
EXTRA_FLAGS = "-arch i386"
QMAKE_LFLAGS += $$EXTRA_FLAGS

but then I end up in the Makefile with:
LFLAGS = -arch i386 -arch\ i386

which will not always work depending on how the linker is called.

Any advice on how to resolve the above issues would be *extremely* appreciated.

Thank you in advance.
Kind regards
JY

[ChrisW67]

Each QMake variable is a list. The regex line in your example works on each value in the list in isolation. When you do not quote the system() output the list gets a populated with elements by splitting on white space. The "-arch" and "arg" go into separate list elements, neither of which matches your regex :(

When you quote (or $$quote()) the system call output you put a single value in the list. Your qmake regex now works but anywhere the single list entry is output into a command line the spaces will be quoted as you have seen.

I'd be inclined to do this on UNIX-ish systems:


MYSQLIBS = $$system(mysql_config --libs | sed -r 's/-arch +[a-z0-9_]+//g')
LIBS += MYSQLIBS

to remove the -arch arg before qmake sees it.

However: you can also use the undocumented split() function thus:

MYSQLIBS = $$quote($$system(mysql_config --libs))
MYSQLIBS ~= s/-arch +[a-z0-9_]+//g #remove all "-arch arg" settings
MYSQLIBS = $$split(MYSQLIBS," ")
LIBS += $$MYSQLIBS



For the QMAKE_LFLAGS problem. I see the same effect on Linux when unquoted and I am guessing that internally the unique() function is used on the QMAKE_LFLAGS variable. When I do this:


EXTRA_FLAGS = "-arch i386"
QMAKE_LFLAGS += $$EXTRA_FLAGS

the second "-arch i386" completely disappears from the Makefile.

Why do you need to duplicate this entry? Will the linker accept "-arch:386" or "-arch=386"?

[jyavenard]

Thanks for the answer !!

Each QMake variable is a list. The regex line in your example works on each value in the list in isolation. When you do not quote the system() output the list gets a populated with elements by splitting on white space. The "-arch" and "arg" go into separate list elements, neither of which matches your regex :(


This is what I've figured out by looking of the outcomes.... Hence why I quoted the bit



I'd be inclined to do this on UNIX-ish systems:


MYSQLIBS = $$system(mysql_config --libs | sed -r 's/-arch +[a-z0-9_]+//g')
LIBS += MYSQLIBS

to remove the -arch arg before qmake sees it.


Yeah, I just didn't want to assume that sed exists and is installed. I now mysql is as it is built as part of my project...



However: you can also use the undocumented
split() function thus:

MYSQLIBS = $$quote($$system(mysql_config --libs))
MYSQLIBS ~= s/-arch +[a-z0-9_]+//g #remove all "-arch arg" settings
MYSQLIBS = $$split(MYSQLIBS," ")
LIBS += $$MYSQLIBS




I hadn't tried the split function, for the time being I had used the sprintf one, which did what I wanted. I assumed the output would never contain text like %1.
But your solution does look nicer.



For the QMAKE_LFLAGS problem. I see the same effect on Linux when unquoted and I am guessing that internally the unique() function is used on the QMAKE_LFLAGS variable. When I do this:


EXTRA_FLAGS = "-arch i386"
QMAKE_LFLAGS += $$EXTRA_FLAGS

the second "-arch i386" completely disappears from the Makefile.



In that case, why does the qmake documentation makes a special mentioned of *= when really, they both do the same thing: += does remove *some* duplicates (not all, as the -arch does stay)


Why do you need to duplicate this entry? Will the linker accept "-arch:386" or "-arch=386"?

I am not actually duplicating the options myself. It just can happen under some circumstances that both are added.

In the configure script generating the package, it retrieves the options for including a series of module.
The output for each module could be -arch i386 -Lpath/to/module -lmodule

All those outputs are added to QMAKE_LFLAG... Unfortunately, the += screws up my list by selectively removing some options (i386 is removed, but -arch does stay)

I still consider this a poorly documented and/or buggy behaviour of qmake, one that isn't portable across all platforms.

Unfortunately, -arch=i386 or "-arch i386" doesn't work, it needs to be two separate arguments provided to ld for it to work.

So at this stage, I'm not sure what I could do to get around the problem, in a way that it will always work for ever cases I could encounter...

Usually compilation in this case will end up failing as there are more options behind the -arch i386.
so as the i386 is removed, I get the error: Error, invalid architecture -Wl,isysroot...
as the LFLAG contains -arch i386 -arch -Wl,isysroot...

Added after 21 minutes:

I found actually something that breaks any assumptions that the code will behave the same on all platforms it is run :(

It actually varies according to which Qt variable you are using:


EXTRA_LIBS = "-arch i386"
LIBS += $$EXTRA_LIBS
QMAKE_LFLAGS += $$EXTRA_LIBS


the result is:


LFLAGS = -arch x86_64 -arch i386 -arch x86_64 -arch x86_64 -Wl,-Bsymbolic-functions -rdynamic -L/usr/lib/mysql -lmysqlclient -Wl,-O1
LIBS = $(SUBLIBS) -L/usr/lib -arch\ x86_64\ -arch\ x86_64 -Wl,-Bsymbolic-functions\ -rdynamic\ -L/usr/lib/mysql\ -lmysqlclient -lQtGui -lQtCore -lpthread


one is escaped, while the other isn't :(

Using split doesn't work either.


EXTRA_LINK = "-arch x86_64 -arch x86_64"
EXTRA_LINK += "$$system(mysql_config --libs)"
EXTRA_LINK = $$split($$EXTRA_LINK, " ")
LIBS = $$EXTRA_LINK


LIBS is actually empty at the end...
So the escaping doesn't occur at the output of $$system, but instead occurs during the += with a different behaviour when adding to QMAKE_LFLAG and LIBS

very puzzling

Added after 7 minutes:

Not sure if responding to myself is in the right etiquette on this forum.

But it seems that rather than adding to LIBS, I should simply add to QMAKE_LFLAGS instead.

When doing += to LFLAGS of a quoted list, it works just fine, escaped characters are de-escaped, and everything works just fine

So my trick here was to do:
QMAKE_LFLAG += $$quote($$VAR)

instead of
QMAKE_LFLAG += $$VAR

JY