PyQt: Proper use of QRunnable and SIGNALs

[chezifresh]

Well managed threading in PyQt can be somewhat challenging. I love QThread for simple background tasks, but when you want to utilize all of the CPUs in a balanced fashion it becomes difficult. You can either create N number of thread instances to do the work, but if you're application has more than one service that wants to make use of background threads at the same time, there can be contention and your machine comes to a halt.

So then there's the QThreadPool, which is a cool idea, but it only accepts QRunnable instances, which do not inherit from QObject, and hence cannot emit SIGNALs. I have a few ideas on how to properly emit SIGNALs from a QRunnable but I also have some concerns about thread safety, and whether or not there's a better way to do this in PyQt. C++ Qt provides nice things like QFuture and QFutureWatcher, but are unsupported in PyQt because of its reliance on C++ templates that apparently do not lend themselves well to Python.

I derive my own class off of QRunnable and create a QObject as a member variable, used for emitting signals. My concern is that, because I'm creating the emitter in the constructor of my runnable, the emitter technically belongs to the thread it was created in. Therefore, if emit a signal with the QObject in the runnable's "run()" method, then I'm probably causing threading issues by calling "emit(...)" on the QObject in a background thread.

Plan A:

class MyRunnable(QtCore.QRunnable):
    def __init__(self):
        QtCore.QRunnable.__init__(self)
        self.emitter = QtCore.QObject()  # 'owned' by the thread it was created in
 
    def run(self):
        # do stuff in a bg thread
 
        # is this bad? emitter technically doesnt emit the signal from the background thread
        # it emits the signal from the thread it was created in.  Also, according to the documentation
        # you cannot call self.emitter.moveToThread(QtCore.QThreadPool.currentThread()) here
        self.emitter.emit(SIGNAL("finshed()"))

class MyRunnable(QtCore.QRunnable):
    def __init__(self):
        QtCore.QRunnable.__init__(self)
        self.emitter = QtCore.QObject()  # 'owned' by the thread it was created in

    def run(self):
        # do stuff in a bg thread

        # is this bad? emitter technically doesnt emit the signal from the background thread
        # it emits the signal from the thread it was created in.  Also, according to the documentation
        # you cannot call self.emitter.moveToThread(QtCore.QThreadPool.currentThread()) here
        self.emitter.emit(SIGNAL("finshed()"))

To copy to clipboard, switch view to plain text mode 

I suppose another option would be to pass in the function you want to connect to and create the emitter in the "run()" method

Plan B:

class MyRunnable(QtCore.QRunnable):
    def __init__(self, callback):
        QtCore.QRunnable.__init__(self)
        self._callback = callback
 
    def run(self):
        self.emitter = QtCore.QObject()
        self.emitter.connect(SIGNAL("finished()"), self._callback)
 
        # do stuff in a bg thread
 
        # thread safe way to connect to the callback
        self.emitter.emit(SIGNAL("finshed()"))

class MyRunnable(QtCore.QRunnable):
    def __init__(self, callback):
        QtCore.QRunnable.__init__(self)
        self._callback = callback

    def run(self):
        self.emitter = QtCore.QObject()
        self.emitter.connect(SIGNAL("finished()"), self._callback)

        # do stuff in a bg thread

        # thread safe way to connect to the callback
        self.emitter.emit(SIGNAL("finshed()"))

To copy to clipboard, switch view to plain text mode 

Is one way better than the other? Is there a better way?

[chezifresh]

I think I can simplify the question a bit.

Is it okay for a QObject to emit a SIGNAL from a different thread than the one it was created in? And are there any consequences for that? ie. will the recipient's SLOT be executed in the thread where the SIGNAL was emitted from?