Multi-threaded GUI possible?

[nurtsi]

Is it possible to open multiple windows with Qt and have a separate rendering thread for each Window? I would like to create multiple windows with each containing a single QGLWidget. Each QGLWidget would then a their own rendering thread that would perform the drawing.

I tried creating multiple QGLWidgets (and QMainWindows) from separate threads, but Qt asserts in QPixmap saying that it should not be used outside the "GUI thread". Since QWidget inherits from QPixmap, it doesn't look like this is possible, but maybe I've missed something?

[tbscope]

The question is not "if" but "why".

First answer the question why you want to draw in different threads?

[high_flyer]

Qt is not designer to have multi threaded rendering.
So if you use Qt "as is", the answer is no.

But are you sure you need rendering in different threads?
I can't think of a case where this should be needed.
You probably need graphical data crunching to be threaded, but the rendering of the data surely can be done in the main gui thread.
And there is no problem doing data crunching in threads, and feed the result to the main thread to be rendered.

Since QWidget inherits from QPixmap

I guess you meant it the other way around...

[nurtsi]

The use case for this is rather special.

The application is visualizing large data sets on a large screen (~30 Megapixels) in real-time using OpenGL. Threaded rendering allows us to circumvent the fact that typically GPUs have only 1 GB of memory which is not enough to fit the visible data set of the entire screen. Our solution to this is to use multiple GPUs and split the screen into tiles (i.e. windows) and have each GPU render only one window which allows us to load only a subset of the entire visible data on one GPU to stay within the memory limits.

Since we use Qt (just not the Gui module) elsewhere in the application, it would be nice if it could handle the window creation and OpenGL setup for us as well so we could avoid having to write the setup code using platform specific native APIs.

[high_flyer]

Hmm...
I still argue that you don't need threaded rendering on the GUI level.
What each GPU is doing in parallel, is still out of Qt's 'realm' - since you are directing in opengl commands to the respective GPU (i guess).
Since you are rendering on multiple screens, you are rendering multiple widgets, and I don't know if this is at all possible to do (in sync) in parallel with out special hardware. (I am not sure the underlying window manager can do parallel window rendering - and maybe you should check that first).

But I don't know if this kind of question and contemplation is out of this forums scope (I mean, it is a valid Qt question, but I doubt any one not very well versed in the details of your system can really answer).
The answer is probably a design issue, rather than implementation detail.
Once you have the correct design (taking Qt abilities in consideration) one can think about how to go and implement it - where threaded rendering may or may not come to be a question...

But, maybe all I have said here is totally wrong, since I know very little about your system.

[wysota]

Application threads (for regular programmers at least) are a concept of CPU not GPU so having multiple threads in the application will not cause the load to be balanced across GPUs and using multiple GPUs doesn't require to have multiple threads in an application. GPU does its job asynchronously to the CPU.

If you have a GL context tied to a specific chip (provided it can be done at all) it should be enough to get that chip to render this particular context. Note you can paint to QGLWidget but you can also render to QGLFramebufferObject or QGLPixelBuffer and then render the results to a widget (or anywhere else).

By the way, here is a (two year old) discussion I found regarding the use of multiple GPUs with OpenGL: http://www.opengl.org/discussion_boa...&Number=235660

[Gus Lott]

GPU does its job asynchronously to the CPU.

Hi Wysota,
I'm not sure that this is entirely accurate. The hardware certainly runs in parallel, but the OpenGL calls block until the operation is completed on the GPU. For example, I'm currently working on a multi-camera application and I need to upload several 640x480 video frames (like 12 or 14) at 30Hz each to the screen. The camera data is coming in unsynchronized, so update calls are sent to the GLWidget such that the rendering function gets called at 60Hz (the monitor refresh rate).

It might also be important for someone to be able to dedicate rendering to a higher speed GLWidget if they were, for example, drawing to a VR projector at 120Hz update and really wanted to make sure that this was all that was being done in the thread.

I know that my calls to upload the new texture on my multi-camera code takes about 400us per call (this scales with the size of the image). So it is true that the GPU is a separate piece of hardware, but the blocking nature of OpenGL calls makes it so that the hardware operations are essentially synchronous. Then the swapbuffer call takes miliseconds to execute.

I have put my GLWidget into a separate rendering thread. I create the GL Widget in the application base thread, spawn a thread, and then use "moveToThread(QThread *) on the GL Widget to a dedicated rendering thread. I have odd results with this command. Sometimes the application allows it. I've got it implemented in such a way that this is the case. You need to make sure that you use the "Qthread::exec()" return in the run method once you move to thread in order to handle the rendering events.

I've profiled this using vtune and it seems like it is really operating in the separate thread, but I'm not sure.

The QGLWidget documentation says that you can Render to a GL context from a separate thread, but you need to use "makeCurrent()" explicitly, etc. I have not tried to go down that route, so it sounds like they've at least thought about it and partially support it in Qt. I would stick with the moveToThread() function on the QGLWidget to achieve rendering in a dedicated thread. That will handle ALL events for that widget in that thread, not just rendering.

Hope that helps

[Gus Lott]

A little more:

From http://doc.qt.nokia.com/4.7/qglwidget.html#details

It is possible to render into a QGLWidget from another thread, but it requires that all access to the GL context is safe guarded. The Qt GUI thread will try to use the context in resizeEvent and paintEvent, so in order for threaded rendering using a GL widget to work, these functions need to be intercepted in the GUI thread and handled accordingly in the application.

So instead of transferring the widget into another thread, you can render from another thread on a timer or something, or intercept the update calls from your GUI thread and send a signal to your render thread.

The render thread just needs to call the makeCurrent() method on your GL widget and then render. You may also have to explicitly swap buffers as well using the swapBuffer() method on the QGLWidget. Make sure you are only rendering from one place at a time. You can get funky results if you have multiple threads sending commands to the GL pipeline for your context and screwing up the order of operations.

I have not tried this rendering from another thread stuff, but it seems supported in the documentation contrary to what others in this thread have said.

[wysota]

The hardware certainly runs in parallel, but the OpenGL calls block until the operation is completed on the GPU.

Certainly not all calls. I'm assuming swapping the framebuffer might be a blocking call, the rest (maybe also apart uploading data from the main memory to the GPU) is probably done asynchronously. Besides if you are rendering many things in the same chip then obviously one frame needs to be finished before the other one may be started.

I have put my GLWidget into a separate rendering thread. I create the GL Widget in the application base thread, spawn a thread, and then use "moveToThread(QThread *) on the GL Widget to a dedicated rendering thread. I have odd results with this command. Sometimes the application allows it. I've got it implemented in such a way that this is the case. You need to make sure that you use the "Qthread::exec()" return in the run method once you move to thread in order to handle the rendering events.

I'm sure this is going to backfire sooner or later. For example when you need to handle some input events in the widget.

The QGLWidget documentation says that you can Render to a GL context from a separate thread, but you need to use "makeCurrent()" explicitly, etc.

Render to a context - yes. But nothing more. QGLWidget is still a QWidget which is not thread-safe and you can't do anything about it. And this all still won't make the application use multiple GPUs.

[Gus Lott]

I'm sure this is going to backfire sooner or later. For example when you need to handle some input events in the widget.

Well, I've reimplemented the mousedown/up/move events, keypress, and the GL drawing/resize/init events in this separate thread on this QGLWidget with a null parent (they're parsed by the thread->exec() loop). I have a separate MainWindow with many other widgets in the primary UI thread handling mouse events and such on QPushButtons and other QGLWidgets.

But when I have to upload about 14 640x480 textures at 60Hz, do rotation/translation/scaling of these polygon mapped textures, and then swapbuffers, these operations begin pushing the 16ms interval that I have at 60Hz monitor refresh rate. Doing much of anything else in that thread (rendering other windows) could prevent this from being achieved.

And I'm pretty sure that most OpenGL calls block until completed. It wouldn't make sense for the pipeline otherwise to have some blocking and some non-blocking. The moment you called another blocking call after several non-blocking calls, the pipeline would have to block until the previous commands had been completed. But I might be wrong on this one. GPUs are highly parallel within the individual openGL calls (i.e. texture mapping and projection calculation), but they're serial in their execution of those calls.

Render to a context - yes. But nothing more. QGLWidget is still a QWidget which is not thread-safe and you can't do anything about it.

I think this is why the documentation says you need to intercept and prevent the draw/resize events in the main UI thread if you're going to draw from another thread.

[DIMEDROLL]

nurtsi,
try this tutorial: http://doc.trolltech.com/qq/qq06-gli...glapplications
Its code is using Qt3, but it can be easily changed for Qt4. I have built a sample project using Qt4.7, so if you need it, let me know and I will attach it here or send via email.
However I have some questions regarding this tutorial, but I'll ask them in separate thread.

[wysota]

And I'm pretty sure that most OpenGL calls block until completed. It wouldn't make sense for the pipeline otherwise to have some blocking and some non-blocking.

I would say exactly the opposite, it makes perfect sense for some calls to by asynchronous and some to be synchronous. Consider the following example pseudocode:

glBegin(...);
glVertex(x1);
glVertex(x2);
glVertex(x3);
glEnd();

glBegin(...);
glVertex(x1);
glVertex(x2);
glVertex(x3);
glEnd();

To copy to clipboard, switch view to plain text mode 

It makes perfect sense to make glVertex calls asynchronous as it makes no sense to transfer data to the graphics card vertex by vertex as you'd be just wasting bandwidth and time. It makes perfect sense to synchronize the context not earlier than the glEnd() call.

The moment you called another blocking call after several non-blocking calls, the pipeline would have to block until the previous commands had been completed.

That's exactly the point Why block at every command if you can only block at the last one? What if there is a context switch somewhere inbetween? What to do with the data already pushed to the pipeline?

I think this is why the documentation says you need to intercept and prevent the draw/resize events in the main UI thread if you're going to draw from another thread.

That's not enough. You can only touch the gl context which is independent of the gl widget but you can't touch the widget itself (i.e. change its geometry) from another thread hence you can't simply push the gl widget to another thread. While it might work in some cases it is bound to fail eventually.

Again, my opinion is you don't need multiple threads to do multi-gpu rendering (of different contexts) and using multiple threads won't make your application do multi-gpu rendering automatically.

[Gus Lott]

I know that that begin/end code actually uploads each vertex at a time (synchronously with all the function entry overhead with each vertex). There's a "glVertex3fv()" call that uploads vertices from an array in memory much more efficiently outside begin/end. Begin and End are not even part of the OpenGL ES subset because of how inefficient individual vertex calls are.

The linear algebra, projections, and vertex culling are done at the swapBuffer call when you're done, I believe. But all the data uploading is synchronous and can take a considerable amount of time.