Passing a big data block from worker thread to GUI

[PeterWurmsdobler]

Hello,

suppose I have a worker thread that carries out some calculations at a certain rate and puts the result into memory, ~16MB, allocated by the main application at startup. Suppose further, a GUI contains a widget that can visualise that data. Since data production will occur at a higher frequency the GUI will be able to display, or that even makes sense to display, I am looking for a mechanism that:

- signals the availability of a new data set to the GUI
- allows the GUI to refresh as fast as it can
- prevents the GUI from blocking the data processing thread
- does not need to copy data unnecessarily

In other words, if the GUI is busy, the newData() signal should bounce off. If the GUI thread is consuming the data, the processing thread should not be impeded.

At the moment I have a QueuedConnection between the processing thread's dataAvailable() slot and the GUI's updateView() slot. The latter then obtains a pointer to the data block in the processing thread object. For now I have not implemented and double buffering.

What is the Qt way of solving this problem?

Kind regards,
peter

[amleto]

You can disconnect the gui slot from the data availability signal every time it starts to process data. This means that if another data set is processed and signaled as ready, the processing gui slot wont be queued up.

However, that on its own will mean that the processing has to copy the data every single time *in case* the gui wants it.

Logically, it probably has to be that way. You dont want a copy to happen whilst the gui is trying to use data. And you shouldnt want the data processing to be so tightly coupled to the gui that it knows whether the gui will be drawing 'this' data set or not...

[PeterWurmsdobler]

Hello and thanks,
I agree, the model should not need to know about the view. I think I might add something in the GUI like:

void DataView::updateView()
{
    if (m_displaying) return;
 
    m_displaying = true;
    float* data = m_processor->getData();
    doTheUpdate(data);
    m_displaying = false;
}

void DataView::updateView()
{
    if (m_displaying) return;
    
    m_displaying = true;
    float* data = m_processor->getData();
    doTheUpdate(data);
    m_displaying = false;
}

To copy to clipboard, switch view to plain text mode 

The updateView() slot should be running in the GUI context as in my case I have used the QueuedConnection. If this is really the case, then the signal-slot mechanism must post the signal into the GUI's event queue which entails that the slot would be processed anyway the next time the GUI has redrawn the View. Having said that, perhaps the previous code is redundant?

What I am after is that the GUI draws the data as often as it can, and with as little delay to the data being available as possible.

I guess in the processor I will use some form of double buffering, such that the getData() method will return a pointer to a data area that is not currently written to. The processor should then manage which area it can write to and which area can be read from. I am not quite sure yet how to do that though.

Regards,
peter

[amleto]

if updateView is a queued slot then m_displaying is redundant - the slot can't be entered concurrently.

You need to be careful that the qt event loop doesnt get saturated if the gui updates are kept locked in 1-to-1 updates with the (faster) data processing. Maybe think about some indexing method so the gui can tell when to skip updates

[PeterWurmsdobler]

Hello and thanks again,

yes, I thought so that m_displaying would be redundant, as the updateView() slot would not be entered twice. Howqever, I wanted to avoid the scenario you mentioned.

I presume if the processing thread fires a signal at a higher rate than the GUI thread can cope with, does this mean that these events will be queued up in the GUI event queue, or will events of the same type be combined into one call of the slot(). If not, how could I set the signal slot mechanism up for this particular connection such that only one event will be taken into account? Could the slot "drain" the event queue for this particular connection?

Cheers,
peter

[amleto]

The number of queued slot calls will just grow, they wont be collapsed.

To deal with this, you'd need an intermediate class to break the coupling.

off the top of my head:
The gui should signal to Intermediate Class (IC) when it is ready for another data set
The processing thread is connected to the IC, not the gui (obviously )
Only when the gui is ready, the IC signals to the gui when a data set is ready.
When the gui is visualising a data set (when it's busy), it signals the IC that it doesnt want updates
The IC should not be in gui thread.

[PeterWurmsdobler]

Hello, thanks for the advice.
As a matter of fact I have already an intermediate class that encapsulates the processing thread and other things. Instead of it forwarding the dataAvailable signal I can make it to be the go between, perhaps with a method enablePublish(bool). It would only re-emit a signal is publishing is enabled by the consumer, the GUI.

Now I can start tackling the data issue, i.e. if the processing thread is writing into a large data block, the consuming thread can only read from another copy.
peter