QPixmap Overhead

[mhoover]

My organization has created some software that displays video by reading frames over UDP and converting the uchar* data into a QImage that gets converted into a QPixmap which we display using setPixmap().

Machines with eight or so cores handle 30 Hz frames nicely, but they tend to get bogged down on conventional CPUs. A single player instance displaying 720x480 frames at 30 Hz ends up occupying 50% of the CPU (this is a dual-core MacBook Pro).

After learning about the performance overhead in converting QImages to QPixmaps (and with some help from the folks here on QtCenter.org) I was able to drop my uchar* data into a QByteArray and convert to a QPixmap. This ends up using about 56% of the CPU –or about 6% more than the QPixmap->QImage conversion.

Instead of creating a separate QByteArray for every frame, I added a QByteArray member to the class that I resize to the size of the header + data. This shaved off about 2% of the cpu expense, but it’s still 4% more costly than the QImage->QPixmap conversion. I’m not sure why that would be?
I’m wondering if there is a way to optimize this process further … could the QByteArray be making a deep copy and slowing things up?
Any guesses?

This thread basically details how I did it:
http://www.qtcentre.org/threads/27998-Creating-a-pixmap-from-unsigned-char*

[^NyAw^]

Hi,

I recommend you to use an QGLWidget and display the images using the GPU. I'm able to reach 30 fps with two cameras using less that 10% CPU, and there is a image processing that consumes some CPU also.
Try seaching google how to display images using OpenGL and use the code into the QGLWidget.

[mhoover]

Wow!

Out of curiosity, how powerful is your CPU/GPU? How many cores?

[^NyAw^]

Hi,

The code was using a Intel P4 3.0GH with Hyperthreading CPU and a NVIDIA 6600GT AGP GPU.

[ChrisW67]

So a CPU is at 100% keeping up with displaying 10,368,000 bytes each second. Are you sure you are only doing the work you think you are? For example are you displaying only when you have a full frame to display and not every time a UDP packet arrives? Are you sure that receiving and assembling the UDP packets is not the bottleneck?

The crude example below sits on 34% (Intel(R) Core(TM)2 CPU 6400 @ 2.13GHz Linux) unless I generate a new page of static every frame using the inefficient method you see. If I display the same static frame every time the CPU is about 7%.


#include <cstdlib>
#include <QtGui>
#include <QDebug>

class mainwindow: public QMainWindow
{
Q_OBJECT
public:
mainwindow(QWidget *p = 0): QMainWindow(p) {
l = new QLabel(this);
setCentralWidget(l);

frame = 0;
QTimer *t = new QTimer(this);
connect(t, SIGNAL(timeout()), this, SLOT(update()));
t->start(33); // 30 times per second
}
public slots:
void update() {
// create a fake frame of video every second frame
if (frame % 2 == 0) {
raw.clear();
for (int i = 0; i < 720 * 480; i++)
raw.append( rand() % 256 );
}
frame++;

pgm.clear();
pgm.append("P5 720 480 255\n");
pgm.append(raw);

QPixmap p;
bool ok = p.loadFromData(pgm, "ppm");
l->setPixmap(p);
};
private:
QLabel *l;
QByteArray raw;
QByteArray pgm;
int frame;
};

int main(int argc, char *argv[])
{
QApplication app(argc, argv);

mainwindow m;
m.show();

return app.exec();
}
#include "main.moc"

[mhoover]

I ended up changing my QLabel to a QGLWidget.

Qt changes all the painting commands to OpenGL functions under the hood with minimal performance cost.

Your suggestion would explain a lot. I should look at it anyway. I don't want to repaint for every packet ...