Combining include files

[marcvanriet]

Hi,

I'm frequently using external libraries (e.g. QExtSerialPort, Qwt, ..). The header files for these libraries need to be in your include path.

Now this list of include directories grows and grows... I would rather put everything in 1 directory, and have 1 header file for each library.

Has anyone ever used a good tool that can combine several include files into 1 big file ?

Best regards,
Marc

[SixDegrees]

In Linux/Unix, this is trivially accomplished at the command line with something like 'cat *.h >> mongo_include.h'.

It is not, however, a good idea. It destroys the modularity of the software involved, and induces a maintenance nightmare of epic proportions. It eliminates portability, as well, and introduces dependencies on external tools to pre-process the build environment.

It may also, in some cases, run afoul of software licenses.

You're much better off either doing standard installations, which place their header and library files in single, known locations, or developing your own consistent installation procedure that does the same. You shouldn't be reaching into multiple directories for header files; if libraries are properly installed, they should all be accessible through some standardized location like /usr/include or /usr/local/include.

[tbscope]

Another technique is to create a wrapper. This means a single include file that includes all the other includes you need.
I'm not sure if that is what you want though. And if you use paths in your includes, it only works on your system and most likely not on another, unless you distribute all the files in such a way that they can always be found via relative paths.

[Timoteo]

Has anyone ever used a good tool that can combine several include files into 1 big file ?

This is a big time no-no. Don't ever create a dependency on a modified (previously) stable header. What you will do is force anyone who attempts to maintain your code to have to do the same modifications on their end, which will (according to Murphy) break every other project in their environment. Seriously don't, just don't.

[marcvanriet]

Hi,

Don't ever create a dependency on a modified (previously) stable header. What you will do is force anyone who attempts to maintain your code to have to do the same modifications on their end, which will (according to Murphy) break every other project in their environment.

I would create a new file, not modify an existing file. Nothing would be broken.

Another technique is to create a wrapper. This means a single include file that includes all the other includes you need.

It was not what I meant, but may be a good alternative. That way I don't have to add all the include directories in my .pro file. Might work if I use relative paths and indeed put all the other files in a consistent location. Just not sure what is used as base path if you use a relative path in a header file. Is it the current project's directory, or the directory of the header file itself. I'll try.

In Linux/Unix, this is trivially accomplished at the command line with something like 'cat *.h >> mongo_include.h'.

That won't work if the header files themselves include header files.

It is not, however, a good idea. It destroys the modularity of the software involved, and induces a maintenance nightmare of epic proportions. It eliminates portability, as well, and introduces dependencies on external tools to pre-process the build environment.

Well, it wouldn't change anything of the software involved. I compile it from source and create its library. It is only for USING this library that I would like a convenient way to have access to the includes.

It may also, in some cases, run afoul of software licenses.

Well, I'm not distributing the library or its source code in any form. Also, I only create personal or in-house software, not something that is going to customers. So licensing should be no problem (don't start a flame war now )

You're much better off either doing standard installations, which place their header and library files in single, known locations, or developing your own consistent installation procedure that does the same. You shouldn't be reaching into multiple directories for header files; if libraries are properly installed, they should all be accessible through some standardized location like /usr/include or /usr/local/include.

Yes, I now that in Linux/Unix everything is just thrown somewhere in /usr/ or /etc or something and it is different for different distributions, and after installing dozens (versions of) libraries you have no idea what is on your system. I rather put the libraries and their include files in a location of my own choice.

I don't understand why this would be a big no-no. I didn't come up with this idea. Sqlite is distributed as an 'amalgamation' and all includes are also put in 1 big "sqlite3.h". Are you all saying that the sqlite people are wrong ?

Best regards,
Marc

[wysota]

I would create a new file, not modify an existing file. Nothing would be broken.

If you upgrade some library it will be a nightmare to check for changes in its header files and compare them against your "new file".

It was not what I meant, but may be a good alternative. That way I don't have to add all the include directories in my .pro file.

No but you will have to include them in other (possibly many) files.

Might work if I use relative paths and indeed put all the other files in a consistent location.

This is asking yourself for trouble. If you copy things, you'll soon end up with several copies of the same files and the already mentioned Murphy's law will make you use the wrong version of the file now and then (possibly when you're just approaching a deadline).

That won't work if the header files themselves include header files.

The only way to merge header files is to use cpp (or equivalent) but then its output depends on the definitions passed to it so it is bound to fail. Just like any attempt to "merge" header files. Consider my classic example:

#ifndef A_H
#define A_H
 
#ifndef X
#define X true
#endif
 
#endif

#ifndef A_H
#define A_H

#ifndef X
#define X true
#endif

#endif

To copy to clipboard, switch view to plain text mode 

#ifndef B_H
#define B_H
 
#ifndef X
#define X false
#endif
 
#endif

#ifndef B_H
#define B_H

#ifndef X
#define X false
#endif

#endif

To copy to clipboard, switch view to plain text mode 

So if you merge files in order a.h, b.h X will be defined to true but if you merge it as b.h, a.h then it will be false and different projects might need different values of X. You just can't process header files aprori to actually compiling the project (with the only exception of precompiled headers which technically does "merge" files but it doesn't process them).

Well, it wouldn't change anything of the software involved.

It would surely make some people lose their hair.

It is only for USING this library that I would like a convenient way to have access to the includes.

If you want convinience then create a file containing needed include directives (like mylibrary.pri for qmake) and include it in your makefile. But stay away from header files themselves.

Well, I'm not distributing the library or its source code in any form.

Which doesn't mean you are not breaking some licence.

Also, I only create personal or in-house software, not something that is going to customers.

Same as above.

So licensing should be no problem (don't start a flame war now )

It's not that easy. If a licence says you can't use the software in some way then you can't use it regardless if you create software for yourself or for someone else. World doesn't end with GPL and LGPL.

Yes, I now that in Linux/Unix everything is just thrown somewhere in /usr/ or /etc or something and it is different for different distributions, and after installing dozens (versions of) libraries you have no idea what is on your system.

You know nothing then. If you install a library, it usually gets its own subdirectory inside /usr/include with version appended to the directory name to avoid conflict.

I rather put the libraries and their include files in a location of my own choice.

And what is stopping you from doing that exactly?

I don't understand why this would be a big no-no. I didn't come up with this idea. Sqlite is distributed as an 'amalgamation' and all includes are also put in 1 big "sqlite3.h". Are you all saying that the sqlite people are wrong ?

SQLite is a very small project, it exports less than 200 symbols and they all fit into a single file because they are always used together. And you are actually wrong that it puts everything in one file. Firstly there is also <sqlite3ext.h> and secondly <sqlite3.h> includes <stdarg.h> which is a standard include file. If id did what you ask for, instead of including stdarg.h it would pull its contents (and all its dependencies) into <sqlite3.h>. So if you upgraded stdarg.h on your system, you'd have to upgrade sqlite3.h as well. What you are trying to do (and the way you are trying to do it) is just a Bad Idea (TM).

[squidge]

Now this list of include directories grows and grows...

So create a project template and use that when you create another project. All include paths already setup with all your other preferred default options.

Secondly, instead of mangling include files, look into pre-compiled include files. They speed up the project compilation stage and if any of the header files change, you can simply rebuild them. If any of projects ever go to someone else (perhaps you want help or its near a dead line and someone else has more time than you) then it'll also work on there system rather than having to say "You'll also need my 12GB AllMyIncludes.h and ensure you have the exact same versions of libraries as I do else it'll segfault"

Combined include files are a nightmare, precompiled include files are a much better idea.

[SixDegrees]

Hi,



I would create a new file, not modify an existing file. Nothing would be broken.



It was not what I meant, but may be a good alternative. That way I don't have to add all the include directories in my .pro file. Might work if I use relative paths and indeed put all the other files in a consistent location. Just not sure what is used as base path if you use a relative path in a header file. Is it the current project's directory, or the directory of the header file itself. I'll try.



That won't work if the header files themselves include header files.



Well, it wouldn't change anything of the software involved. I compile it from source and create its library. It is only for USING this library that I would like a convenient way to have access to the includes.



Well, I'm not distributing the library or its source code in any form. Also, I only create personal or in-house software, not something that is going to customers. So licensing should be no problem (don't start a flame war now )



Yes, I now that in Linux/Unix everything is just thrown somewhere in /usr/ or /etc or something and it is different for different distributions, and after installing dozens (versions of) libraries you have no idea what is on your system. I rather put the libraries and their include files in a location of my own choice.

I don't understand why this would be a big no-no. I didn't come up with this idea. Sqlite is distributed as an 'amalgamation' and all includes are also put in 1 big "sqlite3.h". Are you all saying that the sqlite people are wrong ?

Best regards,
Marc

Well, I'm not interested in arguing for the sake of argument. You've been given these and other reasons why this is a bad idea. If you think it's not, go ahead and do as you like.

[marcvanriet]

..., instead of mangling include files, look into pre-compiled include files. They speed up the project compilation stage and if any of the header files change, you can simply rebuild them.

This is new to me. Looking at qmake precompiled headers it's not really clear to me. It talks about precompiled headers, but the .pro file contains .cpp files also.

How would I use this for e.g. QExtSerialport ? I build the library using the .pro of QExtSerialport and then I have the .lib file. Then what ? In my project create a header file 'stable.h' that includes the QExtSerialport header file(s) and define that as a precompiled header ? Or can the .pch file be put in some common place and be re-used across projects ?

SQLite is a very small project, it exports less than 200 symbols and they all fit into a single file because they are always used together. And you are actually wrong that it puts everything in one file. ... secondly <sqlite3.h> includes <stdarg.h> which is a standard include file. ... if you upgraded stdarg.h on your system, you'd have to upgrade sqlite3.h as well.

I would use it for just this kind of libraries, of which there are dozens available on the internet : QExtSerialPort, Qwt, QSerialDevice, AnalogWidgets, ...

Of course I would only combine the include files of the library itself together, not the standard header files like stdarg.h. Hey, I'm not (that) stupid ! 20 years of programming did learn me some things.

If you want convenience then create a file containing needed include directives (like mylibrary.pri for qmake) and include it in your makefile.

.pri files are new to me also. As far as I understand, they are just files that are included in your .pro file. So how would I use these ? Create a .pri file for QExtSerialPort that adds the necessary include file paths for where I put QExtSerialport. Put this .pri file in the Qt include directory. Reuse this .pri file in different projects of mine ?

Best regards,
Marc

[marcvanriet]

Of course the most convenient way would be if I could just add the name of the library to CONFIG in my .pro file, and all the rest is taken care of automatically.

Like wwWidgets : "Usage of wwWidgets in your projects is trivial. Once you install the package on your system all you have to do is to add the following line to your project (.pro) file: CONFIG+=wwwidgets Just remember to run qmake afterwards. From now on you can use wwWidgets includes and classes in your project."

Regards ,
Marc

[wysota]

This is new to me. Looking at qmake precompiled headers it's not really clear to me. It talks about precompiled headers, but the .pro file contains .cpp files also.

There is no point in precompiling headers if you are not compiling any actual code, right?

How would I use this for e.g. QExtSerialport ?

I think you got it the wrong way. This is not used for projects you already have compiled but rather for projects you wish to compile. And it's not a mechanism for gathering many files into a single file but for speeding up processing header files by the compiler. You still need to have the actual header files should you wish to rebuild the pch file (i.e. when you want to add some more files to the database).

Or can the .pch file be put in some common place and be re-used across projects ?

In theory you can reuse the precompiled header for other projects but it only makes sense if you always use the same set of header files which is rarely the case. Of course unless you like having a several hundred megabyte file lying around on your disk and being read during every compilation cycle (taking up time and memory).

I would use it for just this kind of libraries, of which there are dozens available on the internet : QExtSerialPort, Qwt, QSerialDevice, AnalogWidgets, ...

Well, for Qwt it wouldn't make sense really. Unless you like to have your applications build several times longer. The larger the header file, the longer it takes for the toolchain to preprocess it. We rather want to avoid having to include files than including to many of them. There is a simple test - make a small Qt project consisting just of main() that creates a QString. Then include <QString> compile and measure the compilation time. Then replace <QString> inclusion with <QtCore> and rebuild, measuring time again (just using an appropriate system call and not your stopwatch). If you don't see a significant difference, replace <QtCore> with <QtGui> and add some more includes and then repeat the compilation cycle and measure the time again. Then draw conclusions from your observations and think whether you want that for all your projects.

Of course I would only combine the include files of the library itself together, not the standard header files like stdarg.h. Hey, I'm not (that) stupid

So what makes a "standard header file" different than "include file of the library itself"?

.pri files are new to me also. As far as I understand, they are just files that are included in your .pro file. So how would I use these ? Create a .pri file for QExtSerialPort that adds the necessary include file paths for where I put QExtSerialport.

Yes.

Put this .pri file in the Qt include directory.

No, you can put that anywhere you want as long as you know where it is.

Reuse this .pri file in different projects of mine ?

Yes.

[marcvanriet]

So what makes a "standard header file" different than "include file of the library itself"?

Well, the C standard library header files and Standard C++ header files and the Qt header files etc. In short : all the header files that are not distributed with the library that I wish to use.

Anyway, how do you get it to work in wwWidgets with the CONFIG += wwWidgets ? That looks great !

Regards,
Marc

[wysota]

Well, the C standard library header files and Standard C++ header files and the Qt header files etc. In short : all the header files that are not distributed with the library that I wish to use.

"C standard library" header files are distributed with (g)libc, "Standard C++ header files" are distributed with libstdc++. They are libraries too, just like any other - there is nothing special in them, really.

Anyway, how do you get it to work in wwWidgets with the CONFIG += wwWidgets ? That looks great !

I use feature files for qmake. They are a bit like pri files just slightly more difficult to handle on the deployment end.