A general question about multi-threading (QtConcurrent/QFuture)

[fabietto]

Dear all,

I'd like to ask you a very simple question concerning multi-threading via the QtConcurrent framework.

I'm a complete beginner when it comes to multi-threading. Anyhow, since in one of the application I'm working on right now using more than one thread would be dramatically useful, I've decided to give a try to the QtConcurrent framework.

Could you please tell me if, in your opinion, the few lines of code below, wrote after reading the short tutorials found on the Qt's website, make sense? What I'm trying to do here (of course assuming that PARALLEL_CALCULATIONS has been defined) is to run POPULATION_SIZE instances of the function teams->simTeam[currentTeam]->performAllTests(), each of those inside a separate thread.


for (currentTeam = 0; currentTeam < POPULATION_SIZE; currentTeam++) {

// Evaluate the current team using exploiting or not concurrent programming
#ifdef PARALLEL_CALCULATIONS
synchronizer.addFuture(QtConcurrent::run(teams->simTeam[currentTeam], &Team::performAllTests));
#else
teams->simTeam[currentTeam]->performAllTests();
#endif

}

// Wait until the last thread has ended its execution before moving to the next generation (if parallelized)
#ifdef PARALLEL_CALCULATIONS
synchronizer.waitForFinished();
#endif

Then, I use the synchronizer.waitForFinished() instruction in order to make the program waiting until the execution of all the threads is finished.

Please also consider that the code above is contained inside another for-cycle. Maybe I also need to clear the synchronizer before using it again?

Many thanks to anybody willing to waste his time helping me... :)

Fabio

[wysota]

Maybe something like this is better?

QList<Something> data = teams->simTeam;
QtConcurrent::blockingMapped(data, Team::performAllTests);

[fabietto]

Hey Wisota,

first of all thanks for your very prompt reply! :)

I get a couple of errors when I try to use your code. First of all, the instruction:


QList<Team> data = teams->simTeam;

(Team is the name of the class which teams->simTeam belongs to) doesn't work. The error returned by the compiler is "conversion from Team*[100] to non-scalar type QList<Team> requested" (100 is the number of simTeam present inside teams, stored in an array of pointers). What are you exactly trying to do using that QList instructions?

Then the second instruction prompts the compiler to tell me that blockingMapped is not a member of QtConcurrent. But ok, I'll sort out the latter... :)

[fabietto]

Uhm, apparently I've been able to create the QList, using the following code:


QList<Team*> data;
for (currentTeam = 0; currentTeam < POPULATION_SIZE; currentTeam++)
data << teams->simTeam[currentTeam];

I've also included <QtConcurrentMap> in order to have the blockingMapped function available. Nonetheless, the instruction:


QtConcurrent::blockingMapped(data, Team::performAllTests);

returns me a double error ("invalid use of non-static member function 'void Team:: performAllTests()'" and "no matching function for call to 'blockingMapped(QList<Team*>&, void (Team:: )())'").

Any clues about what I'm doing wrong?

[wysota]

Ok, let's go from the beginning...
If you keep a list of pointers, then the function you pass has to take one argument which has to be a pointer to Team.

void doIt(Team *team) { team->performAllTests(); }
Team* data = teams->simTeam;
QtConcurrent::blockingMapped(data, data+POPULATION_SIZE, doIt);

[fabietto]

Ok, let's go from the beginning...
If you keep a list of pointers, then the function you pass has to take one argument which has to be a pointer to Team.

void doIt(Team *team) { team->performAllTests(); }
Team* data = teams->simTeam;
QtConcurrent::blockingMapped(data, data+POPULATION_SIZE, doIt);

Thanks a lot Wisota, now everything starts to look clearer than before! :)

Still, I have had to use the for cycle, since your instruction for creating the whole QList automatically seems not working (again the error is "conversion from 'Team* [100]' to non-scalar type 'QList<Team*>' requested").

[wysota]

Look at the code again, I'm not creating a list anymore.

[fabietto]

Uhm, still troubles... :-s

I've added a simple function as you were suggesting:


// Function which provide to run a thread
void SimulatorEngine::runThread(Team *team) {

team->performAllTests();

}

Now, when I try to run the blockingMapped() function in this way:


QtConcurrent::blockingMapped(teamsList, teamsList+POPULATION_SIZE, runThread);

(teamsList is a QList<Team*> and POPULATION_SIZE is a constant having value 100) I get an error saying: "no match for 'operator+' in '((SimulatorEngine*)this)->SimulatorEngine::teamsList + 100'".

I reckon that what you intend to do using the three inputs form of blockingMapped() is to provide a range of memory, going from the first element of teamsList to the last one. Even if I use the two parameters syntax, like:


QtConcurrent::blockingMapped(teamsList, runThread);

I still get an error saying: "no matching function for call to 'blockingMapped(QList<Team*>&, <unknown type>)'".

Maybe the runThread() function must be inside the Team class?

[fabietto]

Look at the code again, I'm not creating a list anymore.

Ups, you're right, I was mixing old and new code. Btw, the instruction:


Team* data = teams->simTeam;

returns me another error ("cannot convert 'Team**' to 'Team*' in initialization"). If I'm right, you're trying to make data pointing to the first element of teams->simTeam. I've seen that the instruction:


Team* data = teams->simTeam[0];

compiles smoothly. Do you think that's alright? The problem launching blockingMapped() anyway remains. Even if I create data using the latter instruction, the execution using the syntax:


QtConcurrent::blockingMapped(data, data+POPULATION_SIZE, runThread);

returns a "no matching function for call to 'blockingMapped(Team*&, Team*, <unknown type>)'"

[wysota]

So what is "simTeam"? Team**? Geez... it's C++ not C, man...


#include <QtConcurrentMap>

struct Team {
void perform(){
qDebug("Performing...");
}
};
void doSomething(Team *t){
t->perform();
}

int main(){
QList<Team*> teams;
for(int i=0;i<10;i++)
teams << new Team;
QtConcurrent::blockingMap(teams, doSomething);
qDeleteAll(teams);
qDebug("Raw array...");
Team **teamArr = new Team*[10];
for(int i=0;i<10;i++){
teamArr[i] = new Team;
}

QtConcurrent::blockingMap(teamArr, teamArr+10, doSomething);
for(int i=0;i<10;i++)
delete teamArr[i];
delete [] teamArr;
return 0;
}

[fabietto]

I'm sorry but I'm getting lost, Wisota. Your code still generates errors. The line that provoke this behaviour is again the one where you call QtConcurrent::blockingMap(). If I just keep it as you've showed, the compiler tells me that doSomething is not declared in the scope. If I use the syntax QtConcurrent::blockingMap(teamsList, Team::doSomething) instead, the error returned by g++ concerns the use of a non-static member function.

I'd really like to understand what I'm doing wrong and, overall, what I should do in order to have everything working as expected. I'm perfectly aware that I'm not in the position to ask anything, but if you could please be less cryptic in your replies it would be much easier, quick and helpful to me (as well as less time-wasting to you). Particularly, could I ask you what was wrong with the code that I posted when I opened this thread? And what do you mean exactly when you say "it's C++, not C"?

Anyhow, let's summarise here the main data structures used by my software, hoping this will help:

SimulatorEngine is the main class. From here I'd like to run the multi-threaded code.

SimulatorEngine has a member called teams, pointer to an object belonging to the teamsPopulations class.

teams contains a member called simTeam, which is an array of pointers to objects belonging to the Team class

simTeam contains the function performAllTests, which is highly computational-demanding and therefore it's the one I'd like to execute in separate threads. In pseudo-code it would be something like:


for currentTeam = 0 -> population size
run teams->simTeam[currentTeam]->performAllTests() in multi-threading (simTeam #1 on thread #1, simTeam #2 on thread #2, and so on...)


Basically the idea is to evaluate each simTeam in a separate thread, instead that doing everything sequentially. Since the application will run on a dedicated computer cluster, the benefits deriving from this solution should be significant. Of course, if it might make the things easier, I could move from SimulatorEngine to teamsPopulation in order to run simTeam[currentTeam]->performAllTests().

Many thanks again for your help!

[wysota]

Your code still generates errors.
I compiled it and it works, so it is your code that generates errors :)


If I just keep it as you've showed, the compiler tells me that doSomething is not declared in the scope.
So declare it. It has to be a function, not a method.


If I use the syntax QtConcurrent::blockingMap(teamsList, Team::doSomething) instead, the error returned by g++ concerns the use of a non-static member function.
That's also correct. You can't do it this way because you can only pass a member function of a class that is kept in the container - and you're keeping pointers there, not objects.


I'd really like to understand what I'm doing wrong and, overall, what I should do in order to have everything working as expected.
If you follow my code exactly (without trying to adjust it to your code), it will work.


I'm perfectly aware that I'm not in the position to ask anything, but if you could please be less cryptic in your replies it would be much easier, quick and helpful to me (as well as less time-wasting to you).
I can't be less cryptic. Less cryptic (or more verbose) would only be RTFM.


Particularly, could I ask you what was wrong with the code that I posted when I opened this thread?
Your code was very suboptimal and lengthy. And also resource consuming, but that's a minor problem.


And what do you mean exactly when you say "it's C++, not C"?
Pointers to pointers are not very C++-ish... We have QList to use it.



Anyhow, let's summarise here the main data structures used by my software, hoping this will help:

SimulatorEngine is the main class. From here I'd like to run the multi-threaded code.

SimulatorEngine has a member called teams, pointer to an object belonging to the teamsPopulations class.

teams contains a member called simTeam, which is an array of pointers to objects belonging to the Team class

I would start by getting rid of all those pointers.


Basically the idea is to evaluate each simTeam in a separate thread, instead that doing everything sequentially. Since the application will run on a dedicated computer cluster, the benefits deriving from this solution should be significant. Of course, if it might make the things easier, I could move from SimulatorEngine to teamsPopulation in order to run simTeam[currentTeam]->performAllTests().

My code is a 1:1 mapping to your structures. Just insert it instead of your loop and it will work. My "Team::perform()" is yours "Team::performAllTests()". The rest is the same. If you want to stick with array of pointers, use the version after qDebug("Raw data"). If you're going to swith to using QList, use the version before that statement.

[fabietto]

Ok, Wisota, thanks a lot for the clarifications. I'm still keeping my fingers crossed, but now it looks like everything is fine. I didn't notice (my mistake) that the function you were calling through QtConcurrent wasn't a method but a simple function declared inside the file.

Btw, I'd be more than happy about RTFM. If I only could find a book, a tutorial, an howto or whatsover decently explaining how to use the QtConcurrent framework... :)

[wysota]

Btw, I'd be more than happy about RTFM. If I only could find a book, a tutorial, an howto or whatsover decently explaining how to use the QtConcurrent framework... :)


http://doc.trolltech.com/4.5/threads.html#qtconcurrent-intro
Keeping the GUI Responsive

[fabietto]

http://doc.trolltech.com/4.5/threads.html#qtconcurrent-intro
Keeping the GUI Responsive


In fact... of course I've already seen the documentation about QtConcurrent, but stating that it's bullshit would mean being very polite...