Qt Thread Priorities Under Linux

I need to create a multithreaded application in Qt wherein I am able to assign different thread priorities to different threads.

In general, I want each thread to follow the standard embedded "task" methaphor, where the task pends until it gets a "kick" of some form
(QWaitCondition), after data is loaded into its input queue to process. Each task will process its message(s), and then pend awaiting
another kick upon reception of more data.

Task priorities will be static.

The queues and other collections needs to be thread safe. "wysota" provided source code to a thread safe queue template in the forums.

Since I want each thread to "pend" until it has data, it would seem that I can not use the signals and slots mechanism, as I do NOT
intend to run the Qt event loop exec() in each thread. It has been conveyed that the "exec()" loop does not pend on all platforms.

To the issue of thread priority, the QThread::start() and the QThread::setPriority() methods help text indicates the following:

void QThread::start ( Priority priority = InheritPriority ) [slot]Begins execution of the thread by calling run(), which should be reimplemented in a

QThread subclass to contain your code. The operating system will schedule the thread according to the priority parameter. If the thread is already running, this function does nothing.

The effect of the priority parameter is dependent on the operating system's scheduling policy. In particular, the priority will be ignored on systems that do not support thread priorities (such as on Linux, see http://linux.die.net/man/2/sched_setscheduler for more details).

void QThread::setPriority ( Priority priority )This function sets the priority for a running thread. If the thread is not running, this function does nothing and returns immediately. Use start() to start a thread with a specific priority.

The priority argument can be any value in the QThread::Priority enum except for InheritPriorty.

The effect of the priority parameter is dependent on the operating system's scheduling policy. In particular, the priority will be ignored on systems that do not support thread priorities (such as on Linux, see http://linux.die.net/man/2/sched_setscheduler for more details).

From this information, I would observe the following:

1. It would seem then that I may need to derive from QThread for my threads if I need the ability to set thread priorities.

2. Under Linux, the Qt documentation suggests that there is no support for thread priorities under Linux.

For observation #1, this requirement of deriving from QThread would make helper classes such as QtConcurrent not a good fit for my application.
QThread class have method setPriority(), but QtConcurrent doesn’t. There is a posting available on how to make QtConcurrent accept priorities,
but the finer level of control of QThread begins to look attractive.

For observation #2, this seems to contradict what I observe in the Qt source code for Linux/Unix, and my experience with pthreads.

The following URL (circa 2007) describes problems with thread priorities under Linux.

http://lists.trolltech.com/qt-intere...ad00746-0.html

Does anybody have any recent experience with thread priorities under Linux using Qt?

I think that you need a priority queue rather than threads with different priorities. Since Linux doesn't support priorities for threads that's probably your only option I guess.

Hi wysota, thanks for the quick response.

I should mention, that although we have not yet decided upon an embedded distro (maybe Wind River or MontaVista, or other), we intend to use an embedded Linux distro with real time extensions. I assume that perhaps these extensions might facilitate thread priority support, but I will contact them to verify. I assume the commercial vendors tweak the underlying Linux core as part of their real time offering.

For your suggestions on "priority queues", is there anything inherent to say QQueue for this, or are you referring to a separation by space by using different QQueue's to handle say different message priorities?

When you refer to Linux not having support in it's underlying pthread's, does this apply only to desktop Linux (i.e. Fedora or Ubuntu)?

It's easiest to implement a priority queue using QList with qUpperBound(). As for RT Linux capabilities - as far as I remember (I can be wrong though) threads in Linux are implemented in user space so real-time extensions wouldn't help much. Despite that the main question is whether you really want priorities in threads - do your tasks have priorities or do you just want them to use less resources than the main thread? Furthermore are those tasks time-consuming or rather short-lived?

So if I understand correctly, to your 1st point, the qUpperBound() performs a binary search over the list stopping when a certain threshold is matched.

I assume the binary search might be a binary bisection, and therefore faster than say a linear search?

So the idea here is to assign a queue element a priority at the time of appending to the list in the producer, and then iterate from high to low in the consumer. Is there any downside to a QList vs. a QQueue as the collection class?

Our application would be a user space application with all long lived threads, nothing dynamic. Some of the embedded commercial vendors are suggesting that they might support differing task priorities, but I need to confirm this.

Looking into the Qt source code, it appears that the user argument on thread priority is being passed to the underlying POSIX threads, but the Qt documentation and other user experiences seems to suggest that the
priority value is somewhat meaningless. Of course, this might apply to desktop Linux only, and not the customized kernels tailored for real time Linux.

I think it does suggest, that modelling our final product on say desktop Linux prior to selecting an embedded board could be problematic, or at least we know there will be a lack of fidelity on task priorites and other.

Thanks for the suggestions as always ... wysota to the rescue again!

---

Added after 4 minutes:

I did not completely answer your question, we have a need to prioritize tasks based on what they do, and the data they handle. It is not a matter of saving resources. There will be high priority data to low priority data. For example, the priority of a task that writes to the USB will be lower than say one that writes content to the Ethernet or serial interfaces.

Quote:

---

Originally Posted by bob2oneil

So if I understand correctly, to your 1st point, the qUpperBound() performs a binary search over the list stopping when a certain threshold is matched.

---

Yes, something like that.

Quote:

---

I assume the binary search might be a binary bisection, and therefore faster than say a linear search?

---

Yes, it is O(logn) instead of O(n). But it requires the array to be sorted.

Quote:

---

So the idea here is to assign a queue element a priority at the time of appending to the list in the producer, and then iterate from high to low in the consumer.

---

Correct.

Quote:

---

Is there any downside to a QList vs. a QQueue as the collection class?

---

QQueue inherits QList, so no.

Quote:

---

Looking into the Qt source code, it appears that the user argument on thread priority is being passed to the underlying POSIX threads, but the Qt documentation and other user experiences seems to suggest that the
priority value is somewhat meaningless. Of course, this might apply to desktop Linux only, and not the customized kernels tailored for real time Linux.

---

Linux is Linux. It is not a real-time operating system. The real-time capabilities are usually obtained by running the linux kernel as a regular process in a real-time operating system (such as QNX) in a sort of virtual environment. Linux itself is not RTOS, regardless of its architecture.

Quote:

---

I did not completely answer your question, we have a need to prioritize tasks based on what they do, and the data they handle. It is not a matter of saving resources.

---

So it's likely you need a priority queue rather than thread priorities.

Quote:

---

There will be high priority data to low priority data. For example, the priority of a task that writes to the USB will be lower than say one that writes content to the Ethernet or serial interfaces.

---

These are short-lived tasks so again it seems like a job for a priority queue.

A sort of related question (perhaps not the best segway), but in reading your response to another threading question, you demonstrated in code that an event loop was required in a secondary thread for signals and slots to be operative across threads.

I have a main GUI thread that will be running the Qt event loop via exec().

Per one of our previous conversations, I intend to have secondary threads that pend, and therefore will NOT be running the Qt event loop.

I need to pass data from a secondary thread to the main GUI thread, which presumably will be always active in its event loop.

The secondary thread is then responsible for emitting a signal, which underneath the Qt covers, is really a thread safe event.

Would emitting a signal from the secondary thread (not running an event loop) to a slot in the main GUI thread (running an event loop) be the best method to pass this data?

Alternately, would it be cross thread safe to simply send the data as an event?

Emitting a signal is fine, it's safe and it doesn't require an event loop. Only the receiving side needs to run one.

When doing some threading experimenting under Windows, I seem to recall an instance where a signal was not being received in the Release build, whereas it worked fine in the Debug build.

Have you ever used any implementation of a signal/slot spy to diagnose these types of problems? I was checking out in the Qt Apps a class implementation called "Conan", that looked to be an interesting signal/slot spy.

Is there any particular signal/slot spy that you would recommend?

wyosata, you were kind enough in another thread to provide source code to a thread safe queue you developed. It is my understanding, that this implemenation implements a monitor object, providing thread safe mutual exclusion to the objects it controls.
What is also nice about this C++ template is that it will throttle the producer until the consumer has freed up some queue space.

In my current application, I have several secondary threads, none of which run exec() loops because I want them to pend. The secondary threads will likely have a higher priority than the main GUI thread.

The secondary thread within its thread context will pass data to the primary GUI thread via your thread safe queue.

Do you see any potential priority inversion condition that could occur wherein the GUI thread, having a lower priority than the producer thread, would be indirectly preempted by the lower priority GUI thread effectively "inverting" the relative priorities of the two tasks?

I was doing some reading on how POSIX threads, and speciifically how the Pthread mutex has a provision to handle this condition via priority inheritence or ceiling. However, in looking through the Qt source code, it is clear that Qt's implementation of QMutex does
not have these provisions.

I was wondering if a simple derivation from QMutex might suffice, or in the case of your fine thread safe template, perhaps a derivation from QSemaphore to ensure that a priority inversion solution is in place.

Do you have any thoughts on this?

I'm not sure what you mean, the following sentence seems to be misedited which makes it lose its sense.

Quote:

---

Originally Posted by bob2oneil

Do you see any potential priority inversion condition that could occur wherein the GUI thread, having a lower priority than the producer thread, would be indirectly preempted by the lower priority GUI thread effectively "inverting" the relative priorities of the two tasks?

---

GUI thread would be preemptied by the GUI thread? Could you explain what you really meant?

typo, meant "preempted"

---

Added after 49 minutes:

I must have dyslexia today, let's try this again.

Do you see any potential priority inversion condition that could occur wherein the producer thread, having a higher priority than the GUI thread,
would be indirectly preempted by the lower priority GUI thread effectively "inverting" the relative priorities of the two tasks?

Could this occur if the producer thread pends on say a full queue, while waiting on the GUI consumer thread to empty some content.

I don't see why a low-priority thread would starve a higher-priority thread. Of course it may happen that a high-priority thread is stopped in favour of a low-priority thread but that's an expected and desired situation. Otherwise low-priority threads would never finish their tasks because they'd always be preempted by tasks running with higher priorities. Threads share CPU time and that's the way it should be.

Thanks for your response, I agree with your accessment between the two tasks. However, priority inversion occurs when another medium priority task joins the fray. A definition of this condition is as follows. It could be that it is a non-issue when using your blocking thread safe queue, but I want to make sure I consider all of the possibiliities. In my application, it may be possible for a one to many schenario between queue writers and reader, where for example, the GUI thread will be the singular reader from a thread safe queue, whereas there may be multiple writers, with potentially different thread priorities.

Quote:

---

Consider there is a task L, with low priority. This task requires resource R. Consider that L is running and it acquires resource R. Now, there is another task H, with high priority. This task also requires resource R. Consider H starts after L has acquired resource R. Now H has to wait until L relinquishes resource R.

Everything works as expected up to this point, but problems arise when a new task M starts with medium priority during this time. ` Since R is still in use (by L), H cannot run. Since M is the highest priority unblocked task, it will be scheduled before L. Since L has been preempted by M, L cannot relinquish R. So M will run till it is finished, then L will run - at least up to a point where it can relinquish R - and then H will run. Thus, in above scenario, a task with medium priority ran before a task with high priority, effectively giving us a priority inversion.

In some cases, priority inversion can occur without causing immediate harm—the delayed execution of the high priority task goes unnoticed, and eventually the low priority task releases the shared resource. However, there are also many situations in which priority inversion can cause serious problems. If the high priority task is left starved of the resources, it might lead to a system malfunction or the triggering of pre-defined corrective measures, such as a watch dog timer resetting the entire system. The trouble experienced by the Mars lander "Mars Pathfinder" is a classic example of problems caused by priority inversion in realtime systems.

---

---

Added after 18 minutes:

From the Mars Pathfinder priority inversion solution described at the URL:

http://research.microsoft.com/en-us/...e_Account.html


The solution was to change the creation flags for the semaphore so as to enable priority inheritance.

I don't believe that Qt within the QSemaphore class supports this, anymore than they do for QMutex.

Perhaps this requires different remedies in the application software to specifically avoid priority inversion from happening at all.

Hi wyosta, for your thread safe template, is there a way to specify individually the size of the queue at declaration time? I want to be able to have different queue sizes for different threads/tasks, and I may want to specify
some other user parameters.

So, I have something like the following:

Code:

---

ThreadSafeQueue<QByteArray *>m_queue;/*!< Thread safe message queue */

---

I would like to pass in two arguments to the template such that I can customize some behavior.

Any thoughts?

Quote:

---

Originally Posted by bob2oneil

Thanks for your response, I agree with your accessment between the two tasks. However, priority inversion occurs when another medium priority task joins the fray. A definition of this condition is as follows. It could be that it is a non-issue when using your blocking thread safe queue, but I want to make sure I consider all of the possibiliities. In my application, it may be possible for a one to many schenario between queue writers and reader, where for example, the GUI thread will be the singular reader from a thread safe queue, whereas there may be multiple writers, with potentially different thread priorities.

---

When talking about threads and priorities you can't expect that if you have two threads -- A with high priority and B with low priority and both are ready to be executed then A will always be chosen above B. If that was the case, A could starve B. What priorities mean is that when you have 100% of the time to divide into A and B, A will get more of the CPU time than B.

Quote:

---

Originally Posted by bob2oneil

Hi wyosta, for your thread safe template, is there a way to specify individually the size of the queue at declaration time?

---

You mean the maximum size? Yes, why not. You can modify the template to suit your needs according to what C++ allows.

I guess the 2nd question is what is the syntax in C++ for using your template with a QByteArray *, wherein an argument is passed to override the default maximum size parameter.

My template is nothing special. You can specify size of the queue either as a template argument (as templates can accept value arguments) or as a parameter to the object constructor like for any C++ object.