Reduce build times on Windows

[MarcoAlanen]

Greetings

We are currently having an issue with build times on the Windows platform while the Linux builders can build our project really fast:
Linux: 11m:43s <-- Release
Windows: 57m:37s <-- Debug
Windows: 1h:22m <-- Release

I went through the build log and saw that the moc generation can be extremely slow, especially when it comes to the unit tests. For example:

[01:48:43]: MOC ..\..\PathToTest\TestFile.cpp
[01:48:51]: Compiling...
[01:48:51]: TestFile.cpp

That's a 300 line file using the QTest framework, nothing fancy. Is there any way we can speed up this process or are there some pitfalls we should try to avoid? We're using Qt 4.6.2.

Edit: I found this post which seems to observe the same symptoms but without a solution.

Edit: We tried to disable Windows Defender but it had no impact on the build time.

[stampede]

Hello Marco

Windows: 1h:22m <-- Release

I get quite same build time ( ~1h:30m ) for a qt-based software on windows xp / server, its latest statistics are:
400 source files ( headers + *.cpp, *.c ) ( ~110.000 lines of code )
78 *.ui forms

We are using forward declarations heavily, this reduced build time after updating some header files, as no *.moc recompilation is needed.
I thought that linux builds could be slightly faster, but this difference is really huge.

Is your project of similiar size ?

[MarcoAlanen]

Hello Marco
I get quite same build time ( ~1h:30m ) for a qt-based software on windows xp / server, its latest statistics are:
400 source files ( headers + *.cpp, *.c ) ( ~110.000 lines of code )
78 *.ui forms

We are using forward declarations heavily, this reduced build time after updating some header files, as no *.moc recompilation is needed.
I thought that linux builds could be slightly faster, but this difference is really huge.

Is your project of similiar size ?

616 *.cpp files
844 *.h files
37 *.ui forms

The project isn't really large at all yet and we also rely heavily on forward declarations. Running sloccount gives me these numbers:

Totals grouped by language (dominant language first):
cpp:         120561 (97.64%)
xml:           2529 (2.05%)
python:         235 (0.19%)
sh:             146 (0.12%)

Totals grouped by language (dominant language first):
cpp:         120561 (97.64%)
xml:           2529 (2.05%)
python:         235 (0.19%)
sh:             146 (0.12%)

To copy to clipboard, switch view to plain text mode 

The 8-second example is using a fraction of that code count.

Peculiar indeed

[nish]

are your linux developer using "make -j 2/3/4/..." command?

[MarcoAlanen]

are your linux developer using "make -j 2/3/4/..." command?

We're using -j3 and the builder is dual core IIRC. I can't remember if the Windows builders build in parallel but I think 8 seconds for one small/simple file is too much regardless.
Out of curiosity I did a test run using -j1 in debug mode on my laptop:

real	23m11.487s
user	19m54.867s
sys	2m55.531s

real	23m11.487s
user	19m54.867s
sys	2m55.531s

To copy to clipboard, switch view to plain text mode 

Looks like double the build time. A significant increase in time indeed. However, I can see that the things which take a long time in Windows (moc file generation if I understand it correct) fly by relatively fast at 1-2 seconds per file so in this case I think it's just a matter of under-utilizing the hardware. My laptop with its puny 4200RPM disk is still capable of building faster than the designated build computer

[high_flyer]

Do you use pre compiled headers?

[MarcoAlanen]

Do you use pre compiled headers?

It seems like we don't use them everywhere. Would they speed up the moc generation by a significant amount?

[high_flyer]

Well, it depends on how many QObject you have.
The more QObject you have the more it will improve compilation time, since MOC will not be called for the pre compiled headers.

[MarcoAlanen]

I haven't tried the precompiled headers yet but most of the tools seems to run in molasses when executed in Windows. For the heck of it I updated the toolset to Qt 4.7.1 and discovered that even processing the pro files take a long time:
Windows: 1m 42s
Linux: 7s

It looks like the pro files take anywhere from <0s and up to 10 seconds to process. Each. What the heck is going on?

[high_flyer]

How large is your pro file?

The pro file however, only needs to be processed once, or when some project settings/environment changes.

[MarcoAlanen]

How large is your pro file?

The pro file however, only needs to be processed once, or when some project settings/environment changes.

We have several pro files and a couple (lot) of shared pri files:
$ find ./ | grep -c "\.pro"
69
$ find ./ | grep -c "\.pri"
85

Pro files are in the range of 70 byte to 1.5k with an average of probably around 600b. All in all I'd say not that large and certainly smaller than 1m and 42 seconds

I don't think this is a hardware issue either since I hear a lot of complaints from all our Windows developers. There could perhaps be something fishy in our build scripts. I'll try to verify by comparing the time it takes to generate the build files from Trolltech's own Qt source in Linux and Windows.