PDA

View Full Version : Concurrent execution of QThread or QRunnable (also how to terminate the latter?)



Deusdies
14th May 2012, 23:55
I have the following code:


from PySide.QtCore import *
import time

class GUI(object):

IDLIST = [i for i in xrange(20)]
UNUSEDIDS = [i for i in xrange(20)]

def __init__(self):
print "GUI CLASS INITIALIZED!"

worker = Worker()
worker2 = Worker2()

threadpool = QThreadPool()
threadpool.setMaxThreadCount(10)

for i in xrange(5):
#Alternate between the two
#threadpool.start(worker)
#worker2.start()


@classmethod
def delegator(self):
"""Irrelevant to the question, I need this method for something else"""
USEDIDS = []

toUse = self.UNUSEDIDS[0]
USEDIDS.append(toUse)
self.UNUSEDIDS.pop(0)

return toUse


class Worker(QRunnable):

def __init__(self, parent=None):
super(Worker, self).__init__(parent)

def run(self):
#idInUse = getattr(GUI, "delegator")
idInUse = GUI.delegator()
print "Hello world from QRunnable", idInUse
#time.sleep(5)

class Worker2(QThread):

def __init__(self, parent=None):
super(Worker2, self).__init__(parent)

def run(self):
idInUse = GUI.delegator()
print "Hello world from QThread", idInUse


s = time.time()
GUI()
print "Done in %s" % ((time.time()-s) * 1000)


I think the desired effect is obvious from the code. I want the "Hello world from QThread/QRunnable " to be shown. Since I am writing a multi-threaded application, in my GUI __init__ part I have the loop that starts concurrent threads.

The thing is that, with QRunnable it works just fine. All the 5 threads I specified get executed at once, concurrently. With QThread, however, that is not the case. Instead, I get the following error:


QThread: Destroyed while thread is still running

And it is not executed at all.

Normally I would not at all mind using the QRunnable, however, it does not derive from QObject (so I can't directly emit signals though I can construct a QObject() within it) and also it does not have the .stop() method which I badly need. Googling revealed that there is no way to stop a QRunnable from executing? On the other hand, QThread has both of these methods that I need.

So I guess my question is either how to make multiple same QThreads run concurrently, or how to terminate an execution of a QRunnable?

amleto
15th May 2012, 00:10
why cant you just make and use several QThread instances?

Deusdies
15th May 2012, 00:12
You mean why not make them all separate classes? Well, one because I need them all to be identical (except for some small parts that that Delegator() method decides) and two because I need to create hundreds of such threads.

amleto
15th May 2012, 08:34
no, I said instances, not classes.


for i in xrange(5):
#Alternate between the two
#threadpool.start(worker)
#worker2.start() # no, this is wrong
wrk2 = Worker2() # make a new Worker2.
wrk2.start() # you cant expect 'start()' to work properly if the thread is till running...



You know you dont *need* to inherit from QThread in order to have the work done *in* the thread?

Deusdies
15th May 2012, 13:49
Oh sorry about that - yes I tried that too. The result is:


GUI CLASS INITIALIZED!!!
Done in 3.00002098083
QThread: Destroyed while thread is still running
QThread: Destroyed while thread is still running
QThread: Destroyed while thread is still running
QThread: Destroyed while thread is still running
QThread: Destroyed while thread is still running

If I add a .wait() in that for loop, it works, but then it is not really concurrent.

And yes I am aware that I don't need to inherit from QThread, but I need to... I mean, the rest of the application requires me to (don't ask).

wysota
15th May 2012, 19:38
What if you change "worker2" to "self.worker2" (that is make it an object member)?

Deusdies
15th May 2012, 19:40
Same thing happens :(

amleto
15th May 2012, 22:58
Oh sorry about that - yes I tried that too. The result is:


GUI CLASS INITIALIZED!!!
Done in 3.00002098083
QThread: Destroyed while thread is still running
QThread: Destroyed while thread is still running
QThread: Destroyed while thread is still running
QThread: Destroyed while thread is still running
QThread: Destroyed while thread is still running

If I add a .wait() in that for loop, it works, but then it is not really concurrent.

And yes I am aware that I don't need to inherit from QThread, but I need to... I mean, the rest of the application requires me to (don't ask).

please show your code that does this. If truly is different instances that are being destroyed then it must be the GC, which seems unlikely.

Deusdies
15th May 2012, 23:02
It isn't any different from the one I gave already, except for the instance creation part...



from PySide.QtCore import *
import time

class GUI(object):

IDLIST = [i for i in xrange(20)]
UNUSEDIDS = [i for i in xrange(20)]

def __init__(self):
print "GUI CLASS INITIALIZED!"

worker = Worker()
worker2 = Worker2()

threadpool = QThreadPool()
threadpool.setMaxThreadCount(10)

for i in xrange(5):
#Tried:
#worker2 = Worker2()
#worker2.start()
#...and...
#self.worker2 = Worker2()
#self.worker2.start()


@classmethod
def delegator(self):
"""Irrelevant to the question, I need this method for something else"""
USEDIDS = []

toUse = self.UNUSEDIDS[0]
USEDIDS.append(toUse)
self.UNUSEDIDS.pop(0)

return toUse


class Worker(QRunnable):

def __init__(self, parent=None):
super(Worker, self).__init__(parent)

def run(self):
#idInUse = getattr(GUI, "delegator")
idInUse = GUI.delegator()
print "Hello world from QRunnable", idInUse
#time.sleep(5)

class Worker2(QThread):

def __init__(self, parent=None):
super(Worker2, self).__init__(parent)

def run(self):
idInUse = GUI.delegator()
print "Hello world from QThread", idInUse


s = time.time()
GUI()
print "Done in %s" % ((time.time()-s) * 1000)

amleto
16th May 2012, 00:03
oh, the problem is the way you close the threads! (I think)


close them more politely (call quit() / wait()) instead of ignoring them and letting end of program clean them up.

Deusdies
16th May 2012, 00:21
Now I'm getting some very interesting results.

First run output:


GUI CLASS INITIALIZED!!!
Done in 2.99009998083
QThread: Destroyed while thread is still running
QThread: Destroyed while thread is still running
QThread: Destroyed while thread is still running
QThread: Destroyed while thread is still running
QThread: Destroyed while thread is still running


Second run output:

GUI CLASS INITIALIZED!!!
UNUSEDIDS [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
Hello world from QThread 0
UNUSEDIDS [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
Hello world from QThread 1
QThread: Destroyed while thread is still running
QThread: Destroyed while thread is still running
UNUSEDIDS [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
Hello world from QThread 2
Done in 13.9999389648

Third run output

GUI CLASS INITIALIZED!!!
UNUSEDIDS UNUSEDIDS[ UNUSEDIDS0[ , 0[1, 0, 1, 2, 1, 2, 3UNUSEDIDS, 2, 3, 4[, 3, 04, 5UNUSEDIDS, , 4, Done in 2.00009346008 15, 6
[, , 5, 026, 7, , , 6,


Fourth run output

GUI CLASS INITIALIZED!!!
UNUSEDIDS [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
Hello world from QThread 0
UNUSEDIDS [1, 2, 3, 4, 5, 6, UNUSEDIDS7 , [Done in 3.9999485015981
, , QThread: Destroyed while thread is still running
QThread: Destroyed while thread is still running
92, , 10

Fifth run output the python.exe crashed (Windows displayed the "not responding" message).

What's obvious is that there are different results from every single run. This prompted me to believe that either there's a problem with GC, or that there's an actual bug in Qt.

It crossed my mind to do the following:


self.workers = [Worker2(),Worker2(),Worker2(),Worker2(),Worker2()]
for i in xrange(5):
self.workers[i].start()
time.sleep(0.01)

In other words, sleep for 10 miliseconds after every thread start. Interestingly enough, this (sort-of) worked. Here's the result I got every time after 5 runs:


GUI CLASS INITIALIZED!!!
UNUSEDIDS [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
Hello world from QThread 0
UNUSEDIDS [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
Hello world from QThread 1
UNUSEDIDS [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
Hello world from QThread 2
UNUSEDIDS [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
Hello world from QThread 3
UNUSEDIDS [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
Hello world from QThread 4
Done in 49.9999523163

While not quite as efficient as the QRunnable equivalent (which gets done in 10 ms but this is because the QRunnable's for loop does not have the time.sleep at the end), it's better than nothing. Since my application will be limited to 20 (30) threads at most, the performance impact should be negligible.

I've also noticed that if I reduce the sleep() period to 1 milisecond, the code becomes much more unstable; while it worked fine the first 3 runs, it crashed the interpreter on the 4th run. I am assuming that the higher the value of the sleep is, the more stable the code will be.

amleto
16th May 2012, 08:40
no, you should not be hacking a fix that only addresses the symptoms.

As I say, the problem is at close down - you need to save a handle to all QThreads, and then call wait/quit/terminate on them when you have done everything else in the main thread:




class GUI(object):
def join(self):
for w in self.workers:
w.wait()
...

s = time.time()
g = GUI()
g.join()
print "Done in %s" % ((time.time()-s) * 1000)