Reduce build times on Windows

[MarcoAlanen]

Greetings

We are currently having an issue with build times on the Windows platform while the Linux builders can build our project really fast:
Linux: 11m:43s <-- Release
Windows: 57m:37s <-- Debug
Windows: 1h:22m <-- Release

I went through the build log and saw that the moc generation can be extremely slow, especially when it comes to the unit tests. For example:

[01:48:43]: MOC ..\..\PathToTest\TestFile.cpp
[01:48:51]: Compiling...
[01:48:51]: TestFile.cpp

That's a 300 line file using the QTest framework, nothing fancy. Is there any way we can speed up this process or are there some pitfalls we should try to avoid? We're using Qt 4.6.2.

Edit: I found this post which seems to observe the same symptoms but without a solution.

Edit: We tried to disable Windows Defender but it had no impact on the build time.

[stampede]

Hello Marco

Windows: 1h:22m <-- Release

I get quite same build time ( ~1h:30m ) for a qt-based software on windows xp / server, its latest statistics are:
400 source files ( headers + *.cpp, *.c ) ( ~110.000 lines of code )
78 *.ui forms

We are using forward declarations heavily, this reduced build time after updating some header files, as no *.moc recompilation is needed.
I thought that linux builds could be slightly faster, but this difference is really huge.

Is your project of similiar size ?

[MarcoAlanen]

Hello Marco
I get quite same build time ( ~1h:30m ) for a qt-based software on windows xp / server, its latest statistics are:
400 source files ( headers + *.cpp, *.c ) ( ~110.000 lines of code )
78 *.ui forms

We are using forward declarations heavily, this reduced build time after updating some header files, as no *.moc recompilation is needed.
I thought that linux builds could be slightly faster, but this difference is really huge.

Is your project of similiar size ?

616 *.cpp files
844 *.h files
37 *.ui forms

The project isn't really large at all yet and we also rely heavily on forward declarations. Running sloccount gives me these numbers:

Totals grouped by language (dominant language first):
cpp:         120561 (97.64%)
xml:           2529 (2.05%)
python:         235 (0.19%)
sh:             146 (0.12%)

Totals grouped by language (dominant language first):
cpp:         120561 (97.64%)
xml:           2529 (2.05%)
python:         235 (0.19%)
sh:             146 (0.12%)

To copy to clipboard, switch view to plain text mode 

The 8-second example is using a fraction of that code count.

Peculiar indeed

[nish]

are your linux developer using "make -j 2/3/4/..." command?

[MarcoAlanen]

are your linux developer using "make -j 2/3/4/..." command?

We're using -j3 and the builder is dual core IIRC. I can't remember if the Windows builders build in parallel but I think 8 seconds for one small/simple file is too much regardless.
Out of curiosity I did a test run using -j1 in debug mode on my laptop:

real	23m11.487s
user	19m54.867s
sys	2m55.531s

real	23m11.487s
user	19m54.867s
sys	2m55.531s

To copy to clipboard, switch view to plain text mode 

Looks like double the build time. A significant increase in time indeed. However, I can see that the things which take a long time in Windows (moc file generation if I understand it correct) fly by relatively fast at 1-2 seconds per file so in this case I think it's just a matter of under-utilizing the hardware. My laptop with its puny 4200RPM disk is still capable of building faster than the designated build computer

[high_flyer]

Do you use pre compiled headers?

[MarcoAlanen]

Do you use pre compiled headers?

It seems like we don't use them everywhere. Would they speed up the moc generation by a significant amount?

[high_flyer]

Well, it depends on how many QObject you have.
The more QObject you have the more it will improve compilation time, since MOC will not be called for the pre compiled headers.

[MarcoAlanen]

I haven't tried the precompiled headers yet but most of the tools seems to run in molasses when executed in Windows. For the heck of it I updated the toolset to Qt 4.7.1 and discovered that even processing the pro files take a long time:
Windows: 1m 42s
Linux: 7s

It looks like the pro files take anywhere from <0s and up to 10 seconds to process. Each. What the heck is going on?

[high_flyer]

How large is your pro file?

The pro file however, only needs to be processed once, or when some project settings/environment changes.

[MarcoAlanen]

How large is your pro file?

The pro file however, only needs to be processed once, or when some project settings/environment changes.

We have several pro files and a couple (lot) of shared pri files:
$ find ./ | grep -c "\.pro"
69
$ find ./ | grep -c "\.pri"
85

Pro files are in the range of 70 byte to 1.5k with an average of probably around 600b. All in all I'd say not that large and certainly smaller than 1m and 42 seconds

I don't think this is a hardware issue either since I hear a lot of complaints from all our Windows developers. There could perhaps be something fishy in our build scripts. I'll try to verify by comparing the time it takes to generate the build files from Trolltech's own Qt source in Linux and Windows.

[high_flyer]

I don't see a problem.
If it takes only one sec per pro file, then you are already over a minute.
And it can take few second per pro file - don't forget it needs to be parsed, the paths need to be resolved etc.
Same with the pri files.
Do the math.

A lot of code, compiles longer, simple!

However you don't have to do all the compiling every time, as I said, pro/pri files only compiles once, and if you use precompiled headers you reduce the compile time drastically again.

[MarcoAlanen]

I don't see a problem.
If it takes only one sec per pro file, then you are already over a minute.
And it can take few second per pro file - don't forget it needs to be parsed, the paths need to be resolved etc.
Same with the pri files.
Do the math.

A lot of code, compiles longer, simple!

However you don't have to do all the compiling every time, as I said, pro/pri files only compiles once, and if you use precompiled headers you reduce the compile time drastically again.

Well, in Linux it takes 7 seconds total. I'm trying to figure out why it takes one tenth of the time to build our project in Linux.

And yeah, using precompiled headers and not cleaning will reduce the build times but that's only working around the issue and making it less of a hassle. There must be something fundamentally wrong in our build scripts, Windows, the Qt tools or our hardware for the build time discrepancy to be this large.

[high_flyer]

Well, in Linux it takes 7 seconds total. I'm trying to figure out why it takes one tenth of the time to build our project in Linux.

Windows?

Hmm...
Did you try asking at Nokia about this?

If you really take the same project as is, and get such drastic compile time differences between the two platforms, it may well be a system related (or tools) issue.

[MarcoAlanen]

Nah, not yet. I did search their bug tracker and found nothing of interest regarding my problem which further indicates that the problem is on our end. I have a hunch we're doing something horribly illegal and kind of getting away with it, albeit at a snail's pace.
Now to see how long it takes to do qmake in the Qt source. Those scripts have to be correct.

[nish]

Did you tried to format your same linux machine and install fresh windows on it? Also try with nmake instead of make (of course only if you want to change your compiler on windows).

[MarcoAlanen]

Did you tried to format your same linux machine and install fresh windows on it? Also try with nmake instead of make (of course only if you want to change your compiler on windows).

Nope, haven't tried that. However, we have at least 7 developers who are using Windows and all of them complain about the build times on their computers.

Since I'm a Linux guy and can't really find my way around the Windows world I'll pass this task (including nmake) over to one of the Windows developers. I'll pass this thread along so they can report any findings we have. It might come in handy for someone else in the future.

[MarcoAlanen]

Interesting, I see similar slowdowns in OS X as well on a MacBook Pro with very similar specs (@2.4GHz, 4G RAM) as my Linux laptop.
Running qmake on the Qt-4.7.1 source:
Linux: 1m11.879s
OS X: 4m56.528s

Running qmake on our own project:
Windows: 1m 42s
Linux: 7s
OS X: 46s

For some reason OS X is significantly slower than Linux as well. I'd expect it to be around even if there isn't any fishy stuff in the OS abstraction. Granted, some of this can be caching issues but the OS X and Windows toolset seems to consistently be slower than the Linux ones.