vector parallel processing

[casperbear]

Hi!

I am trying to implement some parallel processing of vector elements using all CPU cores. Is there an easy way to do this in Qt? I.e. some function that takes (for example) a functor and a vector as arguments and spawns threads as necessary.

Of course, one could spawn all threads (one for each element) and start them all. Obviously, this will be not the most effective way in terms of performance and memory usage.

P.S. I am trying to port calls of Parallel.ForEach of .NET Framework 4

[tbscope]

Try QtConcurrent

[casperbear]

Thank you! I was using wrong keywords for searching. Qt word is 'conrurrent', not 'parallel'.

For now I implemented QtConcurrent::run though it's not the approach with the best performance. The strange part is that while concurrent version is somewhat faster than regular version in release build, in debug build it is many times slower. I was able to solve the problem only in this way:

#ifdef _DEBUG
	for (int i = 0; i < Simulation->CompanyVector.size(); i++)
		if (!Simulation->CompanyVector[i]->IsBankrupt)
			MakeDecisionsForCompany(date, Simulation->CompanyVector[i]);
#else
	vector<QFuture<void>> futureVector;
	for (int i = 0; i < Simulation->CompanyVector.size(); i++)
		if (!Simulation->CompanyVector[i]->IsBankrupt)
			futureVector.push_back(QtConcurrent::run(
				this,
				&CArtificialIntelligence::MakeDecisionsForCompany,
				date,
				Simulation->CompanyVector[i]));
	for (int i = 0; i < futureVector.size(); i++)
		futureVector[i].waitForFinished();
#endif

#ifdef _DEBUG
	for (int i = 0; i < Simulation->CompanyVector.size(); i++)
		if (!Simulation->CompanyVector[i]->IsBankrupt)
			MakeDecisionsForCompany(date, Simulation->CompanyVector[i]);
#else
	vector<QFuture<void>> futureVector;
	for (int i = 0; i < Simulation->CompanyVector.size(); i++)
		if (!Simulation->CompanyVector[i]->IsBankrupt)
			futureVector.push_back(QtConcurrent::run(
				this,
				&CArtificialIntelligence::MakeDecisionsForCompany,
				date,
				Simulation->CompanyVector[i]));
	for (int i = 0; i < futureVector.size(); i++)
		futureVector[i].waitForFinished();
#endif

To copy to clipboard, switch view to plain text mode 

[Lesiok]

Because Qt debug code has many asserts especialy with vectors, lists etc.

[tbscope]

The strange part is that while concurrent version is somewhat faster than regular version in release build, in debug build it is many times slower.

If you are using Linux or Unix or MacOS, I can suggest using Systemtap or DTrace for debugging. Performance impact is extremely low. However, these tools are not easy.

[casperbear]

Thank you for your suggestions!
Unfortunately I use only Windows currently.

BTW, I finished initial porting of my project from .NET/C# to Qt/C++. It's quite a straight port (I didn't add or remove any functionality of the simulation part), so I made some performance tests.
Qt/C++ version is 1.33 times faster than .NET/C# version. Both versions used 2 available CPU cores.

And memory consumption was reduced by several times!

If you are interested you can download both versions here
https://sourceforge.net/projects/ittycoon/