How you do *unescape* a string... and some weirdness

[jyavenard]

Hello

In my qmake file, I added to a variable the following:

MYSQLIBS = "$$system(mysql_config --libs)"
MYSQLIBS ~= s/-arch +[a-z0-9_]+//g #remove all "-arch arg" settings
LIBS += MYSQLIBS

Unfortunately here, LIBS in the generated Makefile now contains:
LIBS = $(SUBLIBS) -L/usr/local/mythtv-trunk/lib -Wl,-Bsymbolic-functions\ -rdynamic\ -L/usr/lib/mysql\ -lmysqlclient


see that all the space have been escaped.

What I would like to end-up with is:
LIBS = $(SUBLIBS) -L/usr/local/mythtv-trunk/lib -Wl,-Bsymbolic-functions -rdynamic -L/usr/lib/mysql -lmysqlclient

Otherwise the following would fail to compile under linux.

Now that brings me to another weirdness.
The reason I did:

MYSQLIBS = "$$system(mysql_config --libs)"
instead of:
MYSQLIBS = $$system(mysql_config --libs)

It's because otherwise, the regex replacement that follow would have otherwise no effect at all, and it only works if the variable has been placed in quote.

Another annoyance I do not know how to deal with is: on mac, some linker settings are in the form:
-arch arg
where arg can be i386, x86_64, ppc7400, ppc64

The problem is how qmake deals when adding those parameters.
If I have

EXTRA_LFLAGS = -arch i386
and
QMAKE_LFLAGS = -arch i386

doing:

QMAKE_LFLAGS += $$EXTRA_LFLAGS

The generated makefile will contain:
LFLAGS = -arch i386 -arch

the 2nd i386 has been dropped. Compilation *will* failt
According to the qmake documentation, += shouldn't drop items, that's what *= does:
The *= operator adds a value to the list of values in a variable, but only if it is not already present. This prevents values from being included many times in a variable. For example:

Now I could do:
EXTRA_FLAGS = "-arch i386"
QMAKE_LFLAGS += $$EXTRA_FLAGS

but then I end up in the Makefile with:
LFLAGS = -arch i386 -arch\ i386

which will not always work depending on how the linker is called.

Any advice on how to resolve the above issues would be *extremely* appreciated.

Thank you in advance.
Kind regards
JY

[ChrisW67]

Each QMake variable is a list. The regex line in your example works on each value in the list in isolation. When you do not quote the system() output the list gets a populated with elements by splitting on white space. The "-arch" and "arg" go into separate list elements, neither of which matches your regex

When you quote (or $$quote()) the system call output you put a single value in the list. Your qmake regex now works but anywhere the single list entry is output into a command line the spaces will be quoted as you have seen.

I'd be inclined to do this on UNIX-ish systems:

MYSQLIBS = $$system(mysql_config --libs | sed -r 's/-arch +[a-z0-9_]+//g')
LIBS += MYSQLIBS

MYSQLIBS = $$system(mysql_config --libs | sed -r 's/-arch +[a-z0-9_]+//g')
LIBS += MYSQLIBS

To copy to clipboard, switch view to plain text mode 

to remove the -arch arg before qmake sees it.

However: you can also use the [wiki=Undocumented_qmake#Undocumented_functions]undocumented split()[/wiki] function thus:

MYSQLIBS = $$quote($$system(mysql_config --libs))
MYSQLIBS ~= s/-arch +[a-z0-9_]+//g #remove all "-arch arg" settings
MYSQLIBS = $$split(MYSQLIBS," ")
LIBS += $$MYSQLIBS

MYSQLIBS = $$quote($$system(mysql_config --libs))
MYSQLIBS ~= s/-arch +[a-z0-9_]+//g #remove all "-arch arg" settings
MYSQLIBS = $$split(MYSQLIBS," ")
LIBS += $$MYSQLIBS

To copy to clipboard, switch view to plain text mode 

For the QMAKE_LFLAGS problem. I see the same effect on Linux when unquoted and I am guessing that internally the unique() function is used on the QMAKE_LFLAGS variable. When I do this:

EXTRA_FLAGS = "-arch i386"
QMAKE_LFLAGS += $$EXTRA_FLAGS

EXTRA_FLAGS = "-arch i386"
QMAKE_LFLAGS += $$EXTRA_FLAGS

To copy to clipboard, switch view to plain text mode 

the second "-arch i386" completely disappears from the Makefile.

Why do you need to duplicate this entry? Will the linker accept "-arch:386" or "-arch=386"?

[jyavenard]

Thanks for the answer !!

Each QMake variable is a list. The regex line in your example works on each value in the list in isolation. When you do not quote the system() output the list gets a populated with elements by splitting on white space. The "-arch" and "arg" go into separate list elements, neither of which matches your regex

This is what I've figured out by looking of the outcomes.... Hence why I quoted the bit

I'd be inclined to do this on UNIX-ish systems:

MYSQLIBS = $$system(mysql_config --libs | sed -r 's/-arch +[a-z0-9_]+//g')
LIBS += MYSQLIBS

MYSQLIBS = $$system(mysql_config --libs | sed -r 's/-arch +[a-z0-9_]+//g')
LIBS += MYSQLIBS

To copy to clipboard, switch view to plain text mode 

to remove the -arch arg before qmake sees it.

Yeah, I just didn't want to assume that sed exists and is installed. I now mysql is as it is built as part of my project...

However: you can also use the [wiki=Undocumented_qmake#Undocumented_functions]undocumented
split()[/wiki] function thus:

MYSQLIBS = $$quote($$system(mysql_config --libs))
MYSQLIBS ~= s/-arch +[a-z0-9_]+//g #remove all "-arch arg" settings
MYSQLIBS = $$split(MYSQLIBS," ")
LIBS += $$MYSQLIBS

MYSQLIBS = $$quote($$system(mysql_config --libs))
MYSQLIBS ~= s/-arch +[a-z0-9_]+//g #remove all "-arch arg" settings
MYSQLIBS = $$split(MYSQLIBS," ")
LIBS += $$MYSQLIBS

To copy to clipboard, switch view to plain text mode 

I hadn't tried the split function, for the time being I had used the sprintf one, which did what I wanted. I assumed the output would never contain text like %1.
But your solution does look nicer.

For the QMAKE_LFLAGS problem. I see the same effect on Linux when unquoted and I am guessing that internally the unique() function is used on the QMAKE_LFLAGS variable. When I do this:

EXTRA_FLAGS = "-arch i386"
QMAKE_LFLAGS += $$EXTRA_FLAGS

EXTRA_FLAGS = "-arch i386"
QMAKE_LFLAGS += $$EXTRA_FLAGS

To copy to clipboard, switch view to plain text mode 

the second "-arch i386" completely disappears from the Makefile.

In that case, why does the qmake documentation makes a special mentioned of *= when really, they both do the same thing: += does remove *some* duplicates (not all, as the -arch does stay)

Why do you need to duplicate this entry? Will the linker accept "-arch:386" or "-arch=386"?

I am not actually duplicating the options myself. It just can happen under some circumstances that both are added.

In the configure script generating the package, it retrieves the options for including a series of module.
The output for each module could be -arch i386 -Lpath/to/module -lmodule

All those outputs are added to QMAKE_LFLAG... Unfortunately, the += screws up my list by selectively removing some options (i386 is removed, but -arch does stay)

I still consider this a poorly documented and/or buggy behaviour of qmake, one that isn't portable across all platforms.

Unfortunately, -arch=i386 or "-arch i386" doesn't work, it needs to be two separate arguments provided to ld for it to work.

So at this stage, I'm not sure what I could do to get around the problem, in a way that it will always work for ever cases I could encounter...

Usually compilation in this case will end up failing as there are more options behind the -arch i386.
so as the i386 is removed, I get the error: Error, invalid architecture -Wl,isysroot...
as the LFLAG contains -arch i386 -arch -Wl,isysroot...

---

Added after 21 minutes:

I found actually something that breaks any assumptions that the code will behave the same on all platforms it is run

It actually varies according to which Qt variable you are using:

EXTRA_LIBS = "-arch i386"
LIBS += $$EXTRA_LIBS
QMAKE_LFLAGS += $$EXTRA_LIBS

EXTRA_LIBS = "-arch i386"
LIBS += $$EXTRA_LIBS
QMAKE_LFLAGS += $$EXTRA_LIBS

To copy to clipboard, switch view to plain text mode 

the result is:

LFLAGS        = -arch x86_64 -arch i386 -arch x86_64 -arch x86_64 -Wl,-Bsymbolic-functions -rdynamic -L/usr/lib/mysql -lmysqlclient -Wl,-O1
LIBS          = $(SUBLIBS)  -L/usr/lib -arch\ x86_64\ -arch\ x86_64 -Wl,-Bsymbolic-functions\ -rdynamic\ -L/usr/lib/mysql\ -lmysqlclient -lQtGui -lQtCore -lpthread

LFLAGS        = -arch x86_64 -arch i386 -arch x86_64 -arch x86_64 -Wl,-Bsymbolic-functions -rdynamic -L/usr/lib/mysql -lmysqlclient -Wl,-O1
LIBS          = $(SUBLIBS)  -L/usr/lib -arch\ x86_64\ -arch\ x86_64 -Wl,-Bsymbolic-functions\ -rdynamic\ -L/usr/lib/mysql\ -lmysqlclient -lQtGui -lQtCore -lpthread

To copy to clipboard, switch view to plain text mode 

one is escaped, while the other isn't

Using split doesn't work either.

EXTRA_LINK = "-arch x86_64 -arch x86_64"
EXTRA_LINK += "$$system(mysql_config --libs)"
EXTRA_LINK = $$split($$EXTRA_LINK, " ")
LIBS = $$EXTRA_LINK

EXTRA_LINK = "-arch x86_64 -arch x86_64"
EXTRA_LINK += "$$system(mysql_config --libs)"
EXTRA_LINK = $$split($$EXTRA_LINK, " ")
LIBS = $$EXTRA_LINK

To copy to clipboard, switch view to plain text mode 

LIBS is actually empty at the end...
So the escaping doesn't occur at the output of $$system, but instead occurs during the += with a different behaviour when adding to QMAKE_LFLAG and LIBS

very puzzling

---

Added after 7 minutes:

Not sure if responding to myself is in the right etiquette on this forum.

But it seems that rather than adding to LIBS, I should simply add to QMAKE_LFLAGS instead.

When doing += to LFLAGS of a quoted list, it works just fine, escaped characters are de-escaped, and everything works just fine

So my trick here was to do:
QMAKE_LFLAG += $$quote($$VAR)

instead of
QMAKE_LFLAG += $$VAR

JY