How to display video from raw image data provided by camera SDK

[wo273127571]

Hello, I am developing windows desktop application during my part-time. Now I have a problem which puzzled me for one week.
I can get the camera data from the registered callback function to SDK. Pseudocode show below:


List imageList;//thread safe list
void (int width, int height, void* data)
{
//here I can get the image data, and according to camera fsp N,
//imageList can get N images per second, act as provider
imageList.push(QImage(data));
}

//I use QLabel or QOpenGLWidget to display those images
class QMyLabel : public QLabel
{
void paintEvent(QPaintEvent * event)
{
QImage img = imageList.pop();
painter.drawImage(img);
}

QTimer timer = new QTimer(this);
connect(timer, SIGNAL(timeout()), this, SLOT(update()));
timer.start(60 / N);
}


But it display frame by frame, not smoothly. I have used std::share_ptr and std::move to avoid copy, but still can't fix it.
I Googled it but can't find a solution? Could anyone can give some tips?

[d_stranz]

Why don't you post some actual code instead of a fragment of a class definition that won't even compile? And I sure hope that you mistyped and that QTimer instantiation code isn't actually supposed to be inside the paintEvent().

[wo273127571]

Sorry for this pseudo-code, it's just my mind solution not really implemented code. I'll paste my code later.
But I want to know does my solution is correctly or is a normal solution? Could you give other solutions?

[d_stranz]

If this is the basis for your real code, then I am pretty sure that your problem is coming from your use of a timer. As far as I can see, there is nothing to synchronize the incoming images with the display. Instead you have this timer that fires every 60 / N ms (whatever magic number "N" might be). Maybe you've played with different values of N, trying unsuccessfully to get a smooth update. If N is too big, the timer might fire more frequently than images are being pushed on the list, in which case every pop() that returns an empty QImage results in a flash of black on the label. If N is too small, then you get a list that will grow and grow until you finally run out of memory because the images aren't being popped off fast enough.

You need to synchronize the incoming image flow with the label updates. This means your callback function has to emit a signal or post an event that causes the label to repaint each time a new image arrives. You shouldn't use the list or timer at all. Just send the current image to the label for painting. I think you are using the list as "wishful thinking" - maybe you think the list will shrink and grow. That's not the case. If the painting (or timer) is faster than the rate of incoming images, the list will always have 0 or 1 entry. If the painting (or timer) isn't fast enough, the list will only grow because images will be pushed faster than they are popped.

If the painting is fast enough, then the label will update as fast as the images come in and you get a smooth display. If the painting can't keep up, then images will get dropped. If that is the case, then you don't want to accumulate images on a list, because you will eventually crash and burn when you run out of memory. If the label can't be painted fast enough, it will never paint fast enough to catch up, and the list just grows and grows.

If the images are coming in from the camera really slowly, then you will get a jerky appearance no matter what unless the camera is looking at something that doesn't move. If that's your case, you could get fancier and fade one image out while fading the next one in to give a smoother transition. But that gets into problems with timing and magic numbers again.