PDA

View Full Version : Main thread - worker thread communication.



kikapu
21st May 2007, 15:33
Hi to all and great for me to be here! :)

I was previously used to C# and delegates to communicate from Main Form's thread with a worker thread that i spawn from that form and now i use Python and Qt and trying to do the same thing.
I have the very simple example of counting from "start" to "end" and just display the progress in a progress bar.
The routine that does that is running on a secondary thread. I have made the Gui with Qt Designer and i have the following code that i wrote:



import sys
import threading
from PyQt4 import QtCore, QtGui
from calculatorform_ui import Ui_CalculatorForm
from ui_countNumbers import Ui_MainWindow
from time import sleep

class CountNumbersForm(QtGui.QMainWindow):
def __init__(self, parent=None):
QtGui.QMainWindow.__init__(self, parent)
self.ui = Ui_MainWindow()
self.ui.setupUi(self)

@QtCore.pyqtSignature("")
def on_start_clicked(self):
t = threading.Thread(self.doCount())
t.start()

def doCount(self):
start = self.ui.spFrom.value()
end = self.ui.spTo.value() + 1

for x in range(start, end):
self.ui.progres.setValue(x)
sleep(1)
app.processEvents()

if __name__ == "__main__":
app = QtGui.QApplication(sys.argv)
count = CountNumbersForm();
count.show()
sys.exit(app.exec_())


(spFrom and spTo are spin boxes, "start" is a button that to press to do the job)


Apart from the fact that progress bar does not updated correctly (it stops updating at some percent and not at 100%), my main problem is that main thread (ie. the main Form) is not so responsive as i thought it would be.
I read somewhere that Qt's Signals and Slots mechanism is working correctly in the cross-thread situation but apparently i have made something wrong and thus program hase that behavior.

Can anyone please help me with this ? How can i reliably communicate from Form's main thread to it's working thread in Qt ?

Thanks a lot for any help!

wysota
21st May 2007, 16:26
You're not using signals and slots here. First of all you should use Qt threads instead of Python threads, otherwise there is a good chance it won't work properly.

In run() method of the thread create an object that inherits QObject and there declare a signal which you then have to emit each time you want to change the progress bar value. Then connect the signal to the setValue slot in the progress bar and start the thread. And forget about processEvents() - you don't need it.

kikapu
21st May 2007, 16:41
You're not using signals and slots here. First of all you should use Qt threads instead of Python threads, otherwise there is a good chance it won't work properly.

But i thought that i used Signals and Slots...Isn't the declaration "def on_start_clicked(self):" just an alternative way to the usual "QtCore.QObject.connect" call ?
I mean, the Signals and Slots mechanism is underneath of both ways, isn't ??



In run() method of the thread create an object that inherits QObject and there declare a signal which you then have to emit each time you want to change the progress bar value. Then connect the signal to the setValue slot in the progress bar and start the thread.


Ok, i will try right know this and i let you know. Should i implement a QThread instead of a Python one ??


And forget about processEvents() - you don't need it.

Ok, i just did (forgot) :)

wysota
21st May 2007, 17:02
Yes, provided that start is a button. Yes, you should use QThreads.

kikapu
21st May 2007, 17:58
You're not using signals and slots here. First of all you should use Qt threads instead of Python threads, otherwise there is a good chance it won't work properly.

In run() method of the thread create an object that inherits QObject and there declare a signal which you then have to emit each time you want to change the progress bar value. Then connect the signal to the setValue slot in the progress bar and start the thread. And forget about processEvents() - you don't need it.

At first, a million thanks for helping me out...

So, i am not sure i am following here..
Do you mean to do something like this:



class ProgressNotifier(QObject):
def __init__(self, start, end):
self.start = start
self.end = end

def notify(self):
# declare signal here

class CountNumbersForm(QtGui.QMainWindow):
def __init__(self, parent=None):
QtGui.QMainWindow.__init__(self, parent)
self.ui = Ui_MainWindow()
self.ui.setupUi(self)

@QtCore.pyqtSignature("")
def on_start_clicked(self):
pn = ProgressNotifier(self.ui.spFrom.value(), self.ui.spTo.value() + 1);
# connect the signal here (the method of pn) to the progress bar "setValue" slot
t = threading.Thread(self.doCount())
t.start()

def doCount(self):
for x in range(start, end):
#emit signal so "setValue
sleep(1)


If not, then how can i code it correctly ??
If yes, why i need a second class to do this ??

Again, a million thanks...:)

jpn
21st May 2007, 18:09
But i thought that i used Signals and Slots...Isn't the declaration "def on_start_clicked(self):" just an alternative way to the usual "QtCore.QObject.connect" call ?
I mean, the Signals and Slots mechanism is underneath of both ways, isn't ??

Connecting Signals and Slots (http://www.riverbankcomputing.com/Docs/PyQt4/pyqt4ref.html#connecting-signals-and-slots)

wysota
21st May 2007, 22:55
You're still not using QThread.

kikapu
22nd May 2007, 09:16
You're still not using QThread.

I think the main "problem" is not if i use or not QThread, i will certainly use it if you say that i have to. The main thing is if my skeleton code that i wrote is correct!
Is it ??


@jpn: Thanks, i already have this guide, i'll check it again. :)

kikapu
22nd May 2007, 11:08
Ok, i read some docs and came up with something like the below. I like the postEvent/customEvent thing although i surely try to make an example with directly use the signals/slots mechanism.

But i have the error "TypeError: 'int' object is not callable" at "t.start()" line...:eek:
Can anyone help with this ?

The code is:


import sys
from PyQt4 import QtCore, QtGui
from calculatorform_ui import Ui_CalculatorForm
from ui_countNumbers import Ui_MainWindow
from time import sleep

class WorkerThread(QtCore.QThread):
def __init__(self, start, end, receiver):
QtCore.QThread.__init__(self)
self.receiver = receiver
self.start = start
self.end = end

def run(self):
for x in range(start, end):
#sleep(1)
event = QCustomEvent(200)
event.setData(x)
QThread.postEvent(self.receiver, event)

class CountNumbersForm(QtGui.QMainWindow):
def __init__(self, parent=None):
QtGui.QMainWindow.__init__(self, parent)
self.ui = Ui_MainWindow()
self.ui.setupUi(self)

@QtCore.pyqtSignature("")
def on_start_clicked(self):
t = WorkerThread(self.ui.spFrom.value(), self.ui.spTo.value() + 1, self);
t.start()

def customEvent(self, event):
if event.type() == 200:
i = event.data()
self.ui.progres.setValue(i)

if __name__ == "__main__":
app = QtGui.QApplication(sys.argv)
count = CountNumbersForm();
count.show()
sys.exit(app.exec_())



Thanks a lot for any help...

wysota
22nd May 2007, 11:12
We won't know unless you implement the thread. The thread needs to emit a signal that will update the progress bar. I don't see that in your code.

kikapu
22nd May 2007, 11:45
We won't know unless you implement the thread. The thread needs to emit a signal that will update the progress bar. I don't see that in your code.

Xmm,...ok, i do this now. But, isn't the postEvent/customEvent thing a viable mechanism to do such things ??

wysota
22nd May 2007, 11:58
You can use custom events but using signals is simpler. They use the event mechanism behind the scenes, so the result is the same, just less coding.

kikapu
22nd May 2007, 13:35
You can use custom events but using signals is simpler. They use the event mechanism behind the scenes, so the result is the same, just less coding.

Ok the...I try to make it your way and i post the revised code below. I get the same error message as before: "TypeError: 'int' object is not callable" at t.start() line.

here is the code:



import sys
from PyQt4 import QtCore, QtGui
from calculatorform_ui import Ui_CalculatorForm
from ui_countNumbers import Ui_MainWindow
from time import sleep

class WorkerThread(QtCore.QThread):
def __init__(self, form, start, end):
QtCore.QThread.__init__(self)
self.start = start
self.end = end
self.form = form

def run(self):
c = C()
QtCore.QObject.connect(form.ui.progressBar, c.update, form, QtCore.SLOT("setValue(int)"))

for x in range(start, end):
c.emit(c.update, x)

class C(QtCore.QObject):
update = QtCore.SIGNAL("update(int)")


class CountNumbersForm(QtGui.QMainWindow):
def __init__(self, parent=None):
QtGui.QMainWindow.__init__(self, parent)
self.ui = Ui_MainWindow()
self.ui.setupUi(self)

@QtCore.pyqtSignature("")
def on_start_clicked(self):
t = WorkerThread(self, self.ui.spFrom.value(), self.ui.spTo.value() + 1);
t.start()


I am very much frustrated here...

kikapu
22nd May 2007, 14:12
After a lot of study, i now coded it more correctly (i think) but strangely enough i get the same error message as before. ('int' object is not callable" at t.start() )

Code is:



import sys
from PyQt4 import QtCore, QtGui
from calculatorform_ui import Ui_CalculatorForm
from ui_countNumbers import Ui_MainWindow
from time import sleep

class WorkerThread(QtCore.QThread):
def __init__(self, start, end):
QtCore.QThread.__init__(self)
self.start = start
self.end = end

def run(self):
for x in range(start, end):
self.emit(QtCore.SIGNAL("updateProgress"), x)


class CountNumbersForm(QtGui.QMainWindow):
def __init__(self, parent=None):
QtGui.QMainWindow.__init__(self, parent)
self.ui = Ui_MainWindow()
self.ui.setupUi(self)

@QtCore.pyqtSignature("")
def on_start_clicked(self):
worker = WorkerThread(self.ui.spFrom.value(), self.ui.spTo.value() + 1);
QtCore.QObject.connect(worker, QtCore.SIGNAL("updateProgress"), self.updateProgress, QtCore.Qt.QueuedConnection)
worker.start()

def updateProgress(self, val):
self.ui.progres.setValue(val)


if __name__ == "__main__":
app = QtGui.QApplication(sys.argv)
count = CountNumbersForm();
count.show()
sys.exit(app.exec_())

wysota
22nd May 2007, 14:15
What is class C?

I can give you a draft of C++ code that does what you want. Then you can translate it into python, as my python skills are a bit stale...


class Emiter : public QObject {
Q_OBJECT
public:
Emiter(QObject *parent=0) : QObject(parent){}
void emitSignal(int v){ emit sig(v); }
signals:
void sig(int);
};
class MyThread : public QThread {
Q_OBJECT
public:
MyThread(QObject *parent =0) : QThread(parent){}
void run() {
Emiter e;
connect(&e, SIGNAL(sig(int)), this, SIGNAL(propagate(int)));
i=0;
while(i<=100){
sleep(1);
e.emitSignal(i++);
}
}
signals:
void propagate(int);
};

//...
MyThread thread;
QProgressBar pbar;
connect(&thread, SIGNAL(propagate(int)), &pbar, SLOT(setValue(int)));
thread.start();
Of course there are other ways to do the same thing, for example the propagate() signal is not needed, you can connect the original signal directly to the progress bar if you have access to it.

kikapu
22nd May 2007, 14:23
What is class C?

There is no class C anymore (see my just above post for a more correct solution). Ok, i try to translate your C code although i think that the last Python version i posted is very much alike...

wysota
22nd May 2007, 15:23
The code you posted is dangerous because you emit a signal from a worker thread on account of an object that is in the gui thread and I have no idea in context of which thread the signal will be emitted and will it be queued or executed directly. My Emiter class makes sure the context of the worker thread is used and the signal is queued and propagated in the context of the GUI thread.

kikapu
22nd May 2007, 16:22
The code you posted is dangerous because you emit a signal from a worker thread on account of an object that is in the gui thread

But isn't the whole idea ?? In C# we use delegates to do that, in Qt i hoped that with Signals and Slots i will be able to do it. I think that Qt automatically find the correct thread to "use".

Your Emiter class is "translated" to Python like this ?

class Emiter(QtCore.QObject):
def __init__(self):
QObject.__init__(self)

def emitSignal(self, val):
self.emit(QtCore.SIGNAL("updateProgress"), x)

And what is the purpose (and implementation as it is not shown how it does it's work) of MyThread:: propagate ? Just to forward the signal ?

I think i am stuck...

wysota
22nd May 2007, 16:41
But isn't the whole idea ?? In C# we use delegates to do that, in Qt i hoped that with Signals and Slots i will be able to do it. I think that Qt automatically find the correct thread to "use".
Yes, but I'm not sure what it will decide when you call a function from an object that lives in another thread. It might work, but I'm not sure, therefore I used this kind of "proxy".


Your Emiter class is "translated" to Python like this ?

class Emiter(QtCore.QObject):
def __init__(self):
QObject.__init__(self)

def emitSignal(self, val):
self.emit(QtCore.SIGNAL("updateProgress"), x)

Looks correct.



And what is the purpose (and implementation as it is not shown how it does it's work) of MyThread:: propagate ? Just to forward the signal ?
Yes, just to forward the signal. As I mentioned, you could do it without it, but then you'd have to have access to the progress bar from within the thread object. That's your choice.

kikapu
22nd May 2007, 19:25
Yes, just to forward the signal. As I mentioned, you could do it without it, but then you'd have to have access to the progress bar from within the thread object. That's your choice.

Yes, i now what you mean, we sometimes use it in .net too. But i am afraid i can not go too far, every solution that i implement, i see this error "TypeError: 'int' object is not callable"

I really feel like an idiot, i have written countless thousands lines of code in my life and i am stucked in 10 lines of this. What can i say...:(

wysota
22nd May 2007, 21:34
Are you sure this is correct?

class WorkerThread(QtCore.QThread):
def __init__(self, start, end):
QtCore.QThread.__init__(self)
self.start = start
self.end = end
You have a variable called start and then you call start as a method. Try using a different name here. Maybe that's the problem.

kikapu
22nd May 2007, 21:58
Are you sure this is correct?

class WorkerThread(QtCore.QThread):
def __init__(self, start, end):
QtCore.QThread.__init__(self)
self.start = start
self.end = end
You have a variable called start and then you call start as a method. Try using a different name here. Maybe that's the problem.

...
.....
.......

What can i say...i am just an idiot...The problem was right just under my nose. That was it! I renamed start to pstart and all working ok.
A MILLION thanks wysota!!!

I will now just experiment with the "proxy" style to send the signal.

Thanks again! :)

kikapu
23rd May 2007, 14:28
I have made an exact (i hope) Python version of your C++ code, i post it below.
The thing seems to be working ok, the progress bar is updated correctly and the main thread (form) remains responsive.
(I do not like the fact that i use 3 classes just to do that but anyway...)

So, we connect the propagate SIGNAl to the Emiter one and thus we get the SLOT setValue of the progress bar to be fired when the SIGNAL is sent.

In a previous version of mine, you have said that in the way i had wrote it was a little dangerous because we might not be sure from what thread the code will be executed.
As i study more closely your code (and my Python implementation of this), what make you sure that in this way the function will be called from the correct thread ?


Here is the code:


import sys
from PyQt4 import QtCore, QtGui
from calculatorform_ui import Ui_CalculatorForm
from ui_countNumbers import Ui_MainWindow
from time import sleep

class Emiter(QtCore.QObject):
def __init__(self):
QtCore.QObject.__init__(self)

def emitSignal(self, val):
self.emit(QtCore.SIGNAL("updateProgress(int)"), val)

class WorkerThread(QtCore.QThread):
def __init__(self):
QtCore.QThread.__init__(self)
self.pstart = 0
self.pend = 0

def run(self):
e = Emiter()
propagate = QtCore.SIGNAL("propagate(int)")
QtCore.QObject.connect(e, QtCore.SIGNAL("updateProgress(int)"), self, propagate)

for x in range(self.pstart, self.pend):
sleep(1)
e.emitSignal(x)

class CountNumbersForm(QtGui.QMainWindow):
def __init__(self, parent=None):
QtGui.QMainWindow.__init__(self, parent)
self.ui = Ui_MainWindow()
self.ui.setupUi(self)

@QtCore.pyqtSignature("")
def on_start_clicked(self):
worker.pstart = count.ui.spFrom.value()
worker.pend = count.ui.spTo.value() + 1
self.ui.progress.setRange(worker.pstart, worker.pend)
worker.start()

def updateProgress(self, val):
self.ui.progress.setValue(val)


if __name__ == "__main__":
app = QtGui.QApplication(sys.argv)
count = CountNumbersForm();
worker = WorkerThread();
QtCore.QObject.connect(worker, QtCore.SIGNAL("propagate(int)"), count.ui.progress.setValue, QtCore.Qt.QueuedConnection)
count.show()
sys.exit(app.exec_())


I mean, if i emit entirely the Emit class and the like and replace WorkerThread class with this:

class WorkerThread(QtCore.QThread):
def __init__(self):
QtCore.QThread.__init__(self)
self.pstart = 0
self.pend = 0

def run(self):
for x in range(self.pstart, self.pend):
sleep(1)
self.emit(QtCore.SIGNAL("updateProgress(int)"), x)



what do you think will go wrong ? I read PyQt4 docs and says that the mechanism is able to "Connections may be made across threads." Isn't it our occassion here and particularly because i use queued connections ?? (QtCore.Qt.QueuedConnection)

Thanks again!

wysota
23rd May 2007, 20:46
If you force queued connections, then all will be ok. The thing that makes me sure my approach will work is that the "emit" object is created (and thus handled) in the worker thread (because it's created in the run() method that is executed in context of the worker thread). This means that if I call a method that emits a signal and both the caller (run()) and the callee (Emit instance) are in the same (worker) thread, I can be sure that the signal will be queued because the receiver is in the GUI thread. Now if the callee (QThread object) was in a different thread than the caller (run()) I don't know how Qt would behave. You can check Qt sources and see for yourself - maybe the trick I made was not needed. You can also have the same effect without the Emit object - just move the QThread object to the thread it represents.

kikapu
23rd May 2007, 21:34
If you force queued connections, then all will be ok. The thing that makes me sure my approach will work is that the "emit" object is created (and thus handled) in the worker thread (because it's created in the run() method that is executed in context of the worker thread). This means that if I call a method that emits a signal and both the caller (run()) and the callee (Emit instance) are in the same (worker) thread, I can be sure that the signal will be queued because the receiver is in the GUI thread. Now if the callee (QThread object) was in a different thread than the caller (run()) I don't know how Qt would behave. You can check Qt sources and see for yourself - maybe the trick I made was not needed. You can also have the same effect without the Emit object - just move the QThread object to the thread it represents.

Ok, i see...very helpfull post indeed. The "queued connections", as i saw it, are a very nice way to keep things simple and not complicated. regarding the Singals and Slots mechanism and the communication over different threads. (In fact, i like it even more that the respective C# stuff)
Is there any "catch" with that ? Performance problems or something else ?

wysota
23rd May 2007, 23:09
They are a bit slower as you have to serialise all the arguments and then recreate them back in the other thread when the slot is executed. And there is of course the latency between emiting the signal and the other thread processing it. Oh... and the other thread has to have an event loop running of course.