QPixmap Overhead

[mhoover]

My organization has created some software that displays video by reading frames over UDP and converting the uchar* data into a QImage that gets converted into a QPixmap which we display using setPixmap().

Machines with eight or so cores handle 30 Hz frames nicely, but they tend to get bogged down on conventional CPUs. A single player instance displaying 720x480 frames at 30 Hz ends up occupying 50% of the CPU (this is a dual-core MacBook Pro).

After learning about the performance overhead in converting QImages to QPixmaps (and with some help from the folks here on QtCenter.org) I was able to drop my uchar* data into a QByteArray and convert to a QPixmap. This ends up using about 56% of the CPU –or about 6% more than the QPixmap->QImage conversion.

Instead of creating a separate QByteArray for every frame, I added a QByteArray member to the class that I resize to the size of the header + data. This shaved off about 2% of the cpu expense, but it’s still 4% more costly than the QImage->QPixmap conversion. I’m not sure why that would be?
I’m wondering if there is a way to optimize this process further … could the QByteArray be making a deep copy and slowing things up?
Any guesses?

This thread basically details how I did it:
http://www.qtcentre.org/threads/2799...unsigned-char*

[^NyAw^]

Hi,

I recommend you to use an QGLWidget and display the images using the GPU. I'm able to reach 30 fps with two cameras using less that 10% CPU, and there is a image processing that consumes some CPU also.
Try seaching google how to display images using OpenGL and use the code into the QGLWidget.

[mhoover]

Wow!

Out of curiosity, how powerful is your CPU/GPU? How many cores?

[^NyAw^]

Hi,

The code was using a Intel P4 3.0GH with Hyperthreading CPU and a NVIDIA 6600GT AGP GPU.

[ChrisW67]

So a CPU is at 100% keeping up with displaying 10,368,000 bytes each second. Are you sure you are only doing the work you think you are? For example are you displaying only when you have a full frame to display and not every time a UDP packet arrives? Are you sure that receiving and assembling the UDP packets is not the bottleneck?

The crude example below sits on 34% (Intel(R) Core(TM)2 CPU 6400 @ 2.13GHz Linux) unless I generate a new page of static every frame using the inefficient method you see. If I display the same static frame every time the CPU is about 7%.

#include <cstdlib>
#include <QtGui>
#include <QDebug>
 
class mainwindow: public QMainWindow 
{
    Q_OBJECT
public:
    mainwindow(QWidget *p = 0): QMainWindow(p) {
        l = new QLabel(this);
        setCentralWidget(l);
 
        frame = 0;
        QTimer *t = new QTimer(this);
        connect(t, SIGNAL(timeout()), this, SLOT(update()));
        t->start(33); // 30 times per second
    }
public slots:
    void update() {
        // create a fake frame of video every second frame
        if (frame % 2 == 0) {
            raw.clear();
            for (int i = 0; i < 720 * 480; i++)
                raw.append( rand() % 256 );
        }
        frame++;
 
        pgm.clear();
        pgm.append("P5 720 480 255\n");
        pgm.append(raw);
 
        QPixmap p;
        bool ok = p.loadFromData(pgm, "ppm");
        l->setPixmap(p);
    };
private:
    QLabel *l;
    QByteArray raw;
    QByteArray pgm;
    int frame;
};
 
int main(int argc, char *argv[])
{
    QApplication app(argc, argv);
 
    mainwindow m;
    m.show();
 
    return app.exec();
}
#include "main.moc"

#include <cstdlib>
#include <QtGui>
#include <QDebug>

class mainwindow: public QMainWindow 
{
    Q_OBJECT
public:
    mainwindow(QWidget *p = 0): QMainWindow(p) {
        l = new QLabel(this);
        setCentralWidget(l);

        frame = 0;
        QTimer *t = new QTimer(this);
        connect(t, SIGNAL(timeout()), this, SLOT(update()));
        t->start(33); // 30 times per second
    }
public slots:
    void update() {
        // create a fake frame of video every second frame
        if (frame % 2 == 0) {
            raw.clear();
            for (int i = 0; i < 720 * 480; i++)
                raw.append( rand() % 256 );
        }
        frame++;

        pgm.clear();
        pgm.append("P5 720 480 255\n");
        pgm.append(raw);

        QPixmap p;
        bool ok = p.loadFromData(pgm, "ppm");
        l->setPixmap(p);
    };
private:
    QLabel *l;
    QByteArray raw;
    QByteArray pgm;
    int frame;
};

int main(int argc, char *argv[])
{
    QApplication app(argc, argv);

    mainwindow m;
    m.show();

    return app.exec();
}
#include "main.moc"

To copy to clipboard, switch view to plain text mode 

[mhoover]

I ended up changing my QLabel to a QGLWidget.

Qt changes all the painting commands to OpenGL functions under the hood with minimal performance cost.

Your suggestion would explain a lot. I should look at it anyway. I don't want to repaint for every packet ...