Qt Multithreaded Design Best Practices

[wysota]

Wow... another thread about doing multi-threaded networking...

[bob2oneil]

Hi wysota, this content is currently about multithreaded networking, but it will expand to include several worker threads (such as USB media writers, etc.), many of which are feed data over the network.

So what are your thoughts?

I did read your response to a similar question referencing the following:

http://doc.trolltech.com/qq/qq27-responsive-guis.html

This article provides some insight, but does not address some of the questions I described.

[TemporalBeing]

Well, to answer your questions...

recent default implementation of QThread::run() I think from Qt4.3 and later, simply calls the exec() to use the signal/slots. So all you should have to do is move your various objects to the various threads you want to use.

That said, I highly suggest looking into using the more advanced pointer calsses Qt offers, namely QSharedPointer and QSharedDataPointer, as they would alleviate the need for any one place in your program to officially delete data.

HTH,

Ben

[bob2oneil]

Thanks Ben, I have been using QPointer<> throughout my code, and will consider your suggestions for the others to have "safe pointers" that become zero upon deletion.

What I do not fully understand is whether or not a thread whose run() method (which simply calls exec()) actually pends? I want my secondary threads to pend until they have data to process, hence
the pending queue requirement.

It would seem that the event loop has to be somewhat "running" in the thread for it to both generate signals and respond to invoked slots. This is atypical of nearly all embedded systems that I have
worked on, where tasks lay dormant until necessary for them to perform work.

There has to be some method to have Qt act more like an embedded system of cooperatives tasks, with atomic queues, pending queues, separate task priorities, etc.

My target hardware is a 1.6 GHz Atom processor, single core, so code techniques that utilize threads running on separate cores does not apply.

[wysota]

Hi wysota, this content is currently about multithreaded networking, but it will expand to include several worker threads (such as USB media writers, etc.), many of which are feed data over the network.

So what are your thoughts?

My thoughts are there is a very good chance you don't need threads for "communication" at all. Threads should be used when they have something to do, not when you have some system resources lying unutilized. Communication is by definition asynchronous and making it synchronous is an artificial thing to do. Using threads to counter this artificiality is another artificial thing to do. So why add overhead and complexity if you can do it the easy and direct way.

Bottom line: Use threads if you intend to keep them busy, use signals, slots and events when you want to be notified when something happens. If you can't use asynchronous communication then provide as little code as possible to emulate it using synchronous communication (i.e. use a thread with blocking calls and emit signals or post events from the thread when something interesting happens).

http://libusb.sourceforge.net/api-1....__asyncio.html

[bob2oneil]

Thanks wysota.

I am a big believer in the KISS principle (keep it simple stupid), and would only use threads if absolutely necessary. Therefore, in general, I agree with your advice. I will have to see if I can achieve our design goals
without threading. I have used STL quite a bit in the past to achieve cross platform C++ support for common multitasking concepts such as semaphores, queues, threads, etc. WxWidgets (not to be confused with your
wwWidgets) provides a nice cross platform wrapper on typical threading concepts and communications. I do not have as clear a vision of the Qt equivalence of these per my questions above. I guess I need some more "learning" on Qt.

[wysota]

I am a big believer in the KISS principle (keep it simple stupid), and would only use threads if absolutely necessary.

I would use threads if they are useful. For networking they are not if the technology you use favours asynchronous approach.

I do not have as clear a vision of the Qt equivalence of these per my questions above.

You have QMutex, QSemaphore, QWaitCondition, QSystemSemaphore and a bunch of auxiliary classes (like QMutexLocker).

[bob2oneil]

Hi wysota.

I have looked at the typical multithreaded primitives in Qt, but where do I find say a semaphore with a timeout, or a waiting queue, or would I build these with other primitives such as mutex/semaphore in combination with a QQueue or timers?
As I mentioned, it would seem from the Qt examples, that you have to share QWaitConditions and mutexes between threads, which seems to break the object oriented design metaphor -- but that is ok if that is the way it has to be done.

I am used to having tasks/threads suspend on a waiting queue until work is available. It would seem to me based on my limited understanding an empirical results, that if you intend to have a thread that uses signals and slots, then you implicitly have to invoke
exec() to have the event loop called. It is not clear to me that within the bowels of exec() whether or not this thread actually pends or not? From my limited testing, having a secondary thread attempt to signal the GUI thread, that the GUI
thread has to be active and its event loop running for it to actually receive a signal. This would seem to imply that threads that need to be provided data via signals from secondary threads can not really pend, but instead have to be active, and therefore the benefit
of threading these is of little value. Perhaps that was your point in your previous message.

Assuming I wanted to construct a thread that does behave more like a typcial task in an embedded system design based on an RTOS, one that pends entirely until it has work to perform, maybe that means I can not use one based on an event loop, and therefore signals and
slots can not be used. That is ok, I just need to determine the rules of the game.

P.S.> By the way, I looked at your wwWidgets site, looks like some real nice UI components, thanks for the fine toolkit. I hope to be able to use them sometime in the future.

[wysota]

I have looked at the typical multithreaded primitives in Qt, but where do I find say a semaphore with a timeout,

What is a 'semaphore with a timeout'? Never heard of such thing.

or a waiting queue,

What's a waiting queue? There is a 'wait condition' primitive I'm aware of but not a "waiting queue".

[bob2oneil]

A semaphore with a timeout is a semaphore wait where you can also specify a 2nd condition, which is the maximum time to wait. It is implemented under Unix/Linux using the sem_timedwait() API.
It is particular useful when you have a task blocking on a semaphore waiting on some data, but you also need the task to do something periodically.

http://linux.die.net/man/3/sem_timedwait

In a previous job, we created a standard template library based cross platform library with support for Windows, Linux, and embedded (Texas Instruments TI DSP BIOS RTOS). There were Windows equivalence for Linux sem_timedwait().

A waiting queue is a queue implementation such that it pends the task until data has been received in the queue. These are very common in the embedded world.

I am not aware of any direct equivalent under Qt, but perhaps similar behavior can be built using primitives.

[JohannesMunk]

Am I missing something?

http://doc.trolltech.com/latest/qsem...l#tryAcquire-2

HIH

Joh

[wysota]

A waiting queue is a queue implementation such that it pends the task until data has been received in the queue. These are very common in the embedded world.

That's what you use wait conditions or semaphores for. A primitive is called a primitive because it's primitive. A 'waiting queue' as you call it is a data structure (in your example a queue) protected with a synchronization primitive (semaphore or wait condition).

And a "semaphore with a timeout" is still a regular semaphore. Unfortunately it's a completely useless (in terms of reliability) mechanism. You can get a timeout at the exact moment the semaphore is raised which prevents you from doing the task you want and can lead to a live-lock.

[bob2oneil]

Good input, and the tryAcquire seems to be equivalent to what I have used in the past. For the semaphore with timeout and the live lock condition, when the task is enabled, I examine the queue condition first and handle this in addition to code associated with the timeout.

Does anyone have a feel for my principle question, which is whether or not a thread, using signals and slots and running its own event loop via exec(), actually pends and it awaken by say a signal invocation?

If not, if I were trying to construct such a task, would I do so only by deriving a custom run() routine which had a wait condition or semaphore or the like (and therefore would need to derive from QThread)?

[wysota]

For the semaphore with timeout and the live lock condition, when the task is enabled, I examine the queue condition first and handle this in addition to code associated with the timeout.

Unless you do the examination within the same critical section as you suspend yourself on the semaphore, the result is unreliable as well (the state can change between the check and testing the result of the check).

Does anyone have a feel for my principle question, which is whether or not a thread, using signals and slots and running its own event loop via exec(), actually pends and it awaken by say a signal invocation?

In my opinion it depends on the event loop implementation used and since Qt uses different implementations on different platforms (plus you can use your own) it is probably impossible to answer this question without studying each implementation.

But if you are asking whether receiving the signal while being suspended on a semaphore wakes the thread then the answer is no, it doesn't.

[bob2oneil]

I thought that was probably the case, my question, to some extent was rhetorical. I could not envision how the event loop could be responsive to signals, or generate slots, without it running to some extent.

Therefore, it would seem that if I want to create a thread whose behavior mimics the "task" metaphor that is typical with nearly all embedded system developments I have been involved with, then I would need
to have my own run() routine, one where I would create my own pend criteria (QWaitCondition, or QSemaphore, or QMutex, etc.). It would also seem that since I can not have exec() invoked as my task's execution
loop, then I can not use signals and slots for these particular threads, and take advantage of the thread safe nature of signals and slots. This would seem to imply the need to derive from QThread, which contradicts
the two URLs referenced at the beginning of my message.

Is this summary correct, or am a missing something?