Reducing latency in mouse moves

[PhilFM]

In our interactive video application we need to process a large number of mouse move events as quickly as possible. Each event takes a significant amount of time to process (up to a second). I want to reduce the delays caused by having to wait for one event to be processed before continuing to the next, which causes undesirable latency in the interaction.

To do this I have added code to the event handler which calls a callback several times through the processing of the event to test whether other relevant events are pending. The idea is that if events are pending, the current event handler will be aborted immediately, allowing the new events to be processed with reduced delay. The problem is that I cannot find a way to access the list of pending events. I thought that QApplication would have a way to iterate through the pending events without processing them. The nearest thing is QApplication::processEvents(), but I don't want to process the events, just have a quick look at them. Is there a solution to this, or another way to achieve the same effect?

[marcel]

I suggest moving the processing in a separate thread, if possible.
1 second sounds like a lot of time to waste in the GUI thread.

Regards

[PhilFM]

Marcel yes that would be the best solution. Then the GUI thread could interrupt the processing thread when a relevant event arrives. Unfortunately it would involve me in quite a major redesign :( I was hoping that there was a way to avoid this. Maybe I'll just have to bite the bullet.

[PhilFM]

OK I've done the redesign now, but a few issues came up...

So the problem is that when the user scrubs around on the timeline of our video application, we want the updates to the images in the latest chosen frame to be as fast as possible. If scrubbing has noticeable delays it becomes really annoying. So we want to service the mouse move events really fast. The issue is that we have to read every frame from disk. As the user scrubs through the frames, each is read from disk (in the same thread) so as a new frame is selected, the application is likely to still be busy loading obsolete image data. If a new frame is selected while an old one is being read, we ideally want to abort reading it and switch to the new one, until the user stops moving for enough time so that we can read and display a frame.

So to the solution as suggested my Marcel: put the code to read the frames from disk into another thread. The main thread sends an event to the disk reading thread when the mouse has moved to another frame. The disk reading thread sends an event back to the main thread once it has successfully read a frame from disk. Now when a new mouse move event comes it should be processed asynchronously and allow the application to detect the change of frame using a shared frame counter with a mutex around it, right? Doesn't work. For some reason events don't get processed at the same time as the frame reading thread is running. My evidence for this is that when I qDebug the activity of the threads, the events are only processed when the disk reading thread is waiting for a new frame to process. I don't know why this is.

My solution to this is to call the event handler directly from the disk reading thread at the same time as it detects the change of frame counter. So it calls QApplication::processEvents(0,0). The problem is that sometimes processEvents() called in this way never returns. I'm not sure why this happens either. Probably something to do with callying processEvents() from inside the customEvent() event handler. Thinking about it it doesn't seem like a very good idea...

Anyway the solution to this problem was to create a third thread which is just in a loop calling processEvents() when it gets woken up by the disk reading thread on a QWaitCondition.

So it works, but I don't know if I have the right design now. Can anyone resolve my questions above or tell me a better way to do this?

[wysota]

I admit I haven't read the whole above post, but I have a quick possible solution to your problems. Make your mouse move event as fast as possible, for example by implementing it like this:




... // constructor
m_timer.setInterval(100);
m_timer.setSingleShot(true);
...

...::mouseMoveEvent(...){
m_timer.stop();
m_timer.start();

}
Connect the timers timeout() signal to something like this:

...::timerTimeout(){
emit mouseMoved(QCursor::pos());
}
Then connect your frame loading routine to the mouseMoved(QPoint) signal. This way you'll be loading frames only if mouse wasn't moved for 100ms. All previous mouse moves will be ignored.

This small change should make a huge difference (you might need to adjust the timer interval to fit your needs).