Any fast way(<10ms) to display 640*480 QImage on QGraphicsScene?

[KiwiRider]

Hello everyone,
First of all, I'm new member on here, so let me say HELLO loudly.

I try to search previous threads, but wasn't successful to find the solution so, I'm end up here to post my question.

I'm using QT to developing real time colour recognisation system and it was so far fine, until I got little performance issue here.

Here, I'm using QMainWindow, QGraphicsView and QGraphicsScene to layout my interface, and I'm having 640*480 colour image from camera interface, then this image converted to QImage for display purpose only.

I tried convert this QImage to QPixmap and addItem to Scene(so it displays) then delete it when my next frame is arrived, it was working but very slow.
Like :
scene->addPixmap(QPixmap::fromImage(capture_frame));

So, I tried use QImage as Scene's background, it was much faster than previous method but background is displaying in tile mode, and some what slow still.
scene->setBackgroundBrush(QBrush::QBrush(capture_frame)) ;

So, now I wonder, is there any better way to display QImage into (or under) QGraphicsScene without adding it as item(because it will only good for every 30ms)

Also I need to display some indication marks over this QImage, so I don't want this QImage overlap existing contents of QGraphicsScene.

Hope to read some good ideas from you guys.

Cheers

KiwiRider

[marcel]

Try overriding QGraphicsView::drawBackground(). It might be faster.

Using setBackgroundBrush was drawing tiles because it was considered as a pattern.

Regards

[high_flyer]

I did this with Qt3, but Qt4 is so different that its not aplicable any more.
The general rule is that you should try to minimize Qt's conversion functions as much as possible.
I would try something along the lines of:
- load the buffer to a QImage
- Set QBrush as texture with the QImage
- use QPainter::fillRect() with the texture brush.

I didn't try this, but I'd expect it to be fast enough.

[KiwiRider]

Try overriding QGraphicsView::drawBackground(). It might be faster.

Using setBackgroundBrush was drawing tiles because it was considered as a pattern.

Regards

Hi Marcel,

Thank you for your reply,

Is this what you mean?

QPainter *painter;
QImage *capture_frame_ptr = &capture_frame;
painter = new QPainter(capture_frame_ptr);
scene->drawBackground(painter,rect());

QPainter *painter;
QImage *capture_frame_ptr = &capture_frame;
painter = new QPainter(capture_frame_ptr);
scene->drawBackground(painter,rect());

To copy to clipboard, switch view to plain text mode 

I got access violation error because drawBackground() is protected member either view and scene.

I think I should learn more about c++ before coding with QT, please let me know how to get around this problem.. or am I misunderstanding it completely?

[KiwiRider]

I did this with Qt3, but Qt4 is so different that its not aplicable any more.
The general rule is that you should try to minimize Qt's conversion functions as much as possible.
I would try something along the lines of:
- load the buffer to a QImage
- Set QBrush as texture with the QImage
- use QPainter::fillRect() with the texture brush.

I didn't try this, but I'd expect it to be fast enough.

Hi, high_flyer,

Thank you for your reply

I tried this way,

QPainter *painter;
painter = new QPainter(); 
painter->fillRect(QRect(0,0,width,height),QBrush::QBrush(capture_frame));

QPainter *painter;
painter = new QPainter(); 
painter->fillRect(QRect(0,0,width,height),QBrush::QBrush(capture_frame));

To copy to clipboard, switch view to plain text mode 

where capture_frame is QImage object,

there was no error in compile but no result neither.

Am I doing it right way?

[JoseTlaseca]

Hello everyone,
First of all, I'm new member on here, so let me say HELLO loudly.

I try to search previous threads, but wasn't successful to find the solution so, I'm end up here to post my question.

I'm using QT to developing real time colour recognisation system and it was so far fine, until I got little performance issue here.

Here, I'm using QMainWindow, QGraphicsView and QGraphicsScene to layout my interface, and I'm having 640*480 colour image from camera interface, then this image converted to QImage for display purpose only.

I tried convert this QImage to QPixmap and addItem to Scene(so it displays) then delete it when my next frame is arrived, it was working but very slow.
Like :
scene->addPixmap(QPixmap::fromImage(capture_frame));

So, I tried use QImage as Scene's background, it was much faster than previous method but background is displaying in tile mode, and some what slow still.
scene->setBackgroundBrush(QBrush::QBrush(capture_frame)) ;

So, now I wonder, is there any better way to display QImage into (or under) QGraphicsScene without adding it as item(because it will only good for every 30ms)

Also I need to display some indication marks over this QImage, so I don't want this QImage overlap existing contents of QGraphicsScene.

Hope to read some good ideas from you guys.

Cheers

KiwiRider

Hello KiwiRider...
first of all... welcome... i'm new in this forum too...

I'm working on an object tracking system and i think i can help you... I use QImage object to get pixel access (required for tracking algorithm) and then paint the image using QPainter an QPixmap, i use 640x480 images sequence too...

Here's the way i get good results (speed in show and refresh)...


In header file declare the pixmap, qimage and things you will need:

#ifndef GUI_TRACKING_H
#define GUI_TRACKING_H
 
//here need to include all the classes you need to use
 
class paintManager;
class gui_tracking;
 
class thread : public QThread//neede to create a thread for load images
{
    public:
        gui_tracking *parent;
 
        thread(gui_tracking *p);
        virtual void run();
};
 
QPixmap *pixmapImagen;
QImage *imageImagen;
paintManager *painter;
bool needToUpdate;
int imageHeight, imageWidth;
 
thread *myThread;
...
 
public:
 
void paintEvent(QPaintEvent *);
void customEvent (QCustomEvent *e);
...
#endif

#ifndef GUI_TRACKING_H
#define GUI_TRACKING_H

//here need to include all the classes you need to use

class paintManager;
class gui_tracking;

class thread : public QThread//neede to create a thread for load images
{
    public:
        gui_tracking *parent;

        thread(gui_tracking *p);
        virtual void run();
};

QPixmap *pixmapImagen;
QImage *imageImagen;
paintManager *painter;
bool needToUpdate;
int imageHeight, imageWidth;

thread *myThread;
...

public:

void paintEvent(QPaintEvent *);
void customEvent (QCustomEvent *e);
...
#endif

To copy to clipboard, switch view to plain text mode 

In the gui_tracking implementation file the constructor look something like this:

imageHeight= 480;
imageWidth = 640;
 
imageImagen = new QImage(imageWidth, imageHeight, 32);
 
pixmapImagen = new QPixmap();
 
painter = new paintManager(pixmapImagen, imageHeight, imageWidth, imageHeight, imageWidth);
 
myThread = new thread(this);
 
needToUpdate = TRUE;

imageHeight= 480;
imageWidth = 640;

imageImagen = new QImage(imageWidth, imageHeight, 32);

pixmapImagen = new QPixmap();

painter = new paintManager(pixmapImagen, imageHeight, imageWidth, imageHeight, imageWidth);

myThread = new thread(this);

needToUpdate = TRUE;

To copy to clipboard, switch view to plain text mode 

My paintEvent looks like this:

void gui_tracking::paintEvent(QPaintEvent *)
{
	QPainter p(frame_Image);
	p.drawPixmap(1,1,*pixmapImagen ,0,0,-1,-1);
}

void gui_tracking::paintEvent(QPaintEvent *)
{
	QPainter p(frame_Image);
	p.drawPixmap(1,1,*pixmapImagen ,0,0,-1,-1);
}

To copy to clipboard, switch view to plain text mode 

I use a QThread with a while loop inside that open an image file and show it, so i need to tell "update my pic!!!" to the main thread...I use an event function for that job... this is (needed below)...

void gui_tracking::customEvent (QCustomEvent * e)
{
    if ( e->type() == QEvent::User )
    {
        painter->paintImage(imageImagen);
        repaint(FALSE);
        needToUpdate = TRUE;
    }
}

void gui_tracking::customEvent (QCustomEvent * e)
{
    if ( e->type() == QEvent::User )
    {
        painter->paintImage(imageImagen);
        repaint(FALSE);
        needToUpdate = TRUE;
    }
}

To copy to clipboard, switch view to plain text mode 

In the thread's while loop i do:

void

parent->imageImagen->load(parent->fileName);
//filename is changing every loop with the name of the next image of the sequence
...
 
void thread::run()
{
   while(condition)
   {
        ...
	parent->imageImagen->load(parent->fileName);
 
        if(parent->needToUpdate)
        {
        	QCustomEvent *evento = new QCustomEvent(QEvent::User);
        	parent->needToUpdate = FALSE;
        	QApplication::postEvent(parent, evento);
        }
   }
 
    cout<<"FINISHED"<<endl;
 
}

parent->imageImagen->load(parent->fileName);
//filename is changing every loop with the name of the next image of the sequence
...

void thread::run()
{
   while(condition)
   {
        ...
	parent->imageImagen->load(parent->fileName);

        if(parent->needToUpdate)
        {
        	QCustomEvent *evento = new QCustomEvent(QEvent::User);
        	parent->needToUpdate = FALSE;
        	QApplication::postEvent(parent, evento);
        }
   }

    cout<<"FINISHED"<<endl;

}

To copy to clipboard, switch view to plain text mode 

For the paintManager class... the header file is something like this:

#ifndef PAINTMANAGER_H
#define PAINTMANAGER_H
 
class QPainter;
class QPixmap;
class QPen;
 
class paintManager
{
    public:
        paintManager(QPixmap *pixmap,int rowsImage,int colsImage,int rowsPixmap,int colsPixmap);
 
        ~paintManager();
 
        void paintImage( QImage *imagen );
 
    private:
        int widthImagen;
        int HeightImagen;
        int widthPixmap;
        int heightPixmap;
 
    QPixmap *pixmap;
    QPainter *pixmapPainter;
    QPen *penUsed;
};
#endif

#ifndef PAINTMANAGER_H
#define PAINTMANAGER_H

class QPainter;
class QPixmap;
class QPen;

class paintManager
{
    public:
        paintManager(QPixmap *pixmap,int rowsImage,int colsImage,int rowsPixmap,int colsPixmap);
        
        ~paintManager();

        void paintImage( QImage *imagen );

    private:
        int widthImagen;
        int HeightImagen;
        int widthPixmap;
        int heightPixmap;
  
    QPixmap *pixmap;
    QPainter *pixmapPainter;
    QPen *penUsed;
};
#endif

To copy to clipboard, switch view to plain text mode 

and the implementation file:

#include <qpainter.h>
#include <qpixmap.h>
#include <qpen.h>
#include "paintmanager.h"
 
paintManager(QPixmap *pixmap,int rowsImage,int colsImage,int rowsPixmap,int colsPixmap)
{
    pixmap=pixmap;
    widthImagen =colsImagen;
    heightImagen=rowsImagen;
    widthPixmap=colsPixmap;
    heightPixmap=rowsPixmap;
 
    pixmapPainter= new QPainter();
    penUsed=new QPen();
 
    pixmap->resize(widthPixmap, HeightPixmap);
    pixmap->fill("white");
    pixmapPainter->begin(pixmap);
}
 
paintManager::~paintManager()
{
  pixmapPainter->end();
  delete pixmapPainter;
  delete penUsed;
}
 
void paintManager::paintImagen(QImage *imagen)
{
  pixmapPainter->drawImage(0,0,*imagen,0,0,-1,-1,0);
}

#include <qpainter.h>
#include <qpixmap.h>
#include <qpen.h>
#include "paintmanager.h"

paintManager(QPixmap *pixmap,int rowsImage,int colsImage,int rowsPixmap,int colsPixmap)
{
    pixmap=pixmap;
    widthImagen =colsImagen;
    heightImagen=rowsImagen;
    widthPixmap=colsPixmap;
    heightPixmap=rowsPixmap;

    pixmapPainter= new QPainter();
    penUsed=new QPen();

    pixmap->resize(widthPixmap, HeightPixmap);
    pixmap->fill("white");
    pixmapPainter->begin(pixmap);
}

paintManager::~paintManager()
{
  pixmapPainter->end();
  delete pixmapPainter;
  delete penUsed;
}

void paintManager::paintImagen(QImage *imagen)
{
  pixmapPainter->drawImage(0,0,*imagen,0,0,-1,-1,0);
}

To copy to clipboard, switch view to plain text mode 

And thats all... that works for me... sorry if there is something wrong... (like a spanish name) my original code is in spanish... but this is the basic idea...
Hope help you... if not... sorry...
Any question contact me...

Salu2!!!


PS: I dont recommend convert QImage to Qpixmap... that's toooooo slow... read the QImage documentation...

[sr105]

JoseTlaseca,

First, thank you for the great reply.

1. How fast are you running this, i.e. how many images are you loading and displaying per second? This will give the rest of us an idea of its performance
2. Would using a QGraphicsView be faster than the QPixmap? It was recommended to me in the #QT IRC channel.
3. Do you have to worry about locking with your needToUpdate and imageImagen?
4. What's the difference between these techniques?
   
   scene->addPixmap(QPixmap::fromImage(capture_frame));
   
   pixmapPainter->drawImage(); p.drawPixmap(1,1,*pixmapImagen ,0,0,-1,-1);
   
   scene = new QGraphicsScene(0, 0, image.width(), image.height());
   view->setScene(scene);
   view->setBackgroundBrush(image);

[JohannesMunk]

The only reason why your original approach is so slow, is because your are adding a new item to the scene over and over again. If you only update the QGraphicsPixmapItem your performance should be good enough.

So save a reference to a QGraphicsPixmapItem and call item->setPixmap(yourNewPixmap) when a new image arrives from your camera.

To prove my statement I wrote a quick and dirty test application, that pre generates a number of pixmaps and shows them over and over again. Skip the details of the ImageSource class. Its just to have something to draw.

It schedules around 800 updates per second. That should be fast enough for your purpose?

main.h
 
#ifndef MAIN_H
#define MAIN_H
 
#include <QtGui>
//#include <QtOpenGL>
 
class ImageSource : QObject
{    Q_OBJECT
public:
    ImageSource(QString text = "Hello!",int w = 640,int h = 480) : m_width(w),m_height(h)
    {
        if (text.length() == 0) text = "Hello!";
        // Create pixmaps for each character
        for (int i=0;i<text.length();++i)
        {
            QString imageText(text.length(),' ');
            imageText[i] = text.at(i);
            QPixmap* pm = new QPixmap(w,h);
            pm->fill(Qt::white);
            QPainter p(pm);
            p.setFont(QFont("Courier"));
            p.setPen(Qt::black);
            p.drawText(QRectF(0,0,w,h),Qt::AlignCenter,imageText);
            dummies.append(pm);
        }
        current = dummies.length();
    }
    QPixmap* nextImage()
    {
        --current;
        if (current < 0)
            current = dummies.length()-1;
        //return dummies.at(3);
        return dummies.at(current);
    }
    int width() {return m_width;}
    int height() {return m_height;}
private:
    QList<QPixmap* > dummies;
    int current;
    int m_width;
    int m_height;
};
 
class MainWindow : public QMainWindow
{    Q_OBJECT
public:
    MainWindow()
    {
        scene = new QGraphicsScene(this);
 
        imageSrc = new ImageSource();
 
        pixmapItem = new QGraphicsPixmapItem();
        scene->addItem(pixmapItem);
 
        view = new QGraphicsView();
        //view->setViewport(new QGLWidget(QGLFormat()));
        view->setScene(scene);
 
        setCentralWidget(view);
 
        QTimer* updateTimer = new QTimer(this);
        QObject::connect(updateTimer,SIGNAL(timeout()),this,SLOT(doUpdate()));
        updateTimer->start(1);
 
        QTimer* fpsTimer = new QTimer(this);
        QObject::connect(fpsTimer,SIGNAL(timeout()),this,SLOT(showFPS()));
        fpsTimer->start(1000);
    }
    ~MainWindow() {}
protected slots:
    void doUpdate()
    {
        pixmapItem->setPixmap(*imageSrc->nextImage());
        ++frameCounter;
    }
    void showFPS()
    {
        setWindowTitle(QString("FPS: %1").arg(frameCounter));
        frameCounter = 0;
    }
 
private:
    QGraphicsScene* scene;
    QGraphicsView* view;
    ImageSource* imageSrc;
    QGraphicsPixmapItem* pixmapItem;
    int frameCounter;
};
 
#endif // MAIN_H
 
 
main.cpp:
 
#include <QtCore>
#include <QtGui>
 
#include "main.h"
 
int main(int argc, char *argv[])
{
    QApplication a(argc, argv);
 
    MainWindow mw;
    mw.show();
 
    return a.exec();
}
 
pro-file:
 
//QT += opengl
TARGET = FastImages
TEMPLATE = app
SOURCES += main.cpp
HEADERS += \
     main.h

main.h

#ifndef MAIN_H
#define MAIN_H

#include <QtGui>
//#include <QtOpenGL>

class ImageSource : QObject
{    Q_OBJECT
public:
    ImageSource(QString text = "Hello!",int w = 640,int h = 480) : m_width(w),m_height(h)
    {
        if (text.length() == 0) text = "Hello!";
        // Create pixmaps for each character
        for (int i=0;i<text.length();++i)
        {
            QString imageText(text.length(),' ');
            imageText[i] = text.at(i);
            QPixmap* pm = new QPixmap(w,h);
            pm->fill(Qt::white);
            QPainter p(pm);
            p.setFont(QFont("Courier"));
            p.setPen(Qt::black);
            p.drawText(QRectF(0,0,w,h),Qt::AlignCenter,imageText);
            dummies.append(pm);
        }
        current = dummies.length();
    }
    QPixmap* nextImage()
    {
        --current;
        if (current < 0)
            current = dummies.length()-1;
        //return dummies.at(3);
        return dummies.at(current);
    }
    int width() {return m_width;}
    int height() {return m_height;}
private:
    QList<QPixmap* > dummies;
    int current;
    int m_width;
    int m_height;
};

class MainWindow : public QMainWindow
{    Q_OBJECT
public:
    MainWindow()
    {
        scene = new QGraphicsScene(this);

        imageSrc = new ImageSource();

        pixmapItem = new QGraphicsPixmapItem();
        scene->addItem(pixmapItem);

        view = new QGraphicsView();
        //view->setViewport(new QGLWidget(QGLFormat()));
        view->setScene(scene);

        setCentralWidget(view);

        QTimer* updateTimer = new QTimer(this);
        QObject::connect(updateTimer,SIGNAL(timeout()),this,SLOT(doUpdate()));
        updateTimer->start(1);

        QTimer* fpsTimer = new QTimer(this);
        QObject::connect(fpsTimer,SIGNAL(timeout()),this,SLOT(showFPS()));
        fpsTimer->start(1000);
    }
    ~MainWindow() {}
protected slots:
    void doUpdate()
    {
        pixmapItem->setPixmap(*imageSrc->nextImage());
        ++frameCounter;
    }
    void showFPS()
    {
        setWindowTitle(QString("FPS: %1").arg(frameCounter));
        frameCounter = 0;
    }

private:
    QGraphicsScene* scene;
    QGraphicsView* view;
    ImageSource* imageSrc;
    QGraphicsPixmapItem* pixmapItem;
    int frameCounter;
};

#endif // MAIN_H


main.cpp:

#include <QtCore>
#include <QtGui>

#include "main.h"

int main(int argc, char *argv[])
{
    QApplication a(argc, argv);

    MainWindow mw;
    mw.show();

    return a.exec();
}

pro-file:

//QT += opengl
TARGET = FastImages
TEMPLATE = app
SOURCES += main.cpp
HEADERS += \
     main.h

To copy to clipboard, switch view to plain text mode 

[KiwiRider]

Thank you JohannesMunk,

It improves alot

I also found the pixmap set previously deleted from memory gracefully.

Cheers

[^NyAw^]

Hi,

Of course, you can use the advantage of using OpenGL to display the images that will increase the performance.

[JohannesMunk]

That's right. I've shown how in my minimal example - just uncomment the setViewPort line. In this case it greatly reduces the CPU-load and the FPS goes as high as the timer schedules ~ 1000 FPS. Its a special situation though, as the same pixmaps are drawn all the time. If a texture object needs to be created for every frame the opengl performance could suffer. I've seen situations with big textures where the opengl engine was slower than the raster engine.

Joh

[^NyAw^]

Hi,

I've been working with a little project that uses two cameras capturing 640x480 images and displaying the images into a openGL widget at 30fps each. Also there was image processing with the captured images. The CPU was arround 14%.

[wysota]

If a texture object needs to be created for every frame the opengl performance could suffer. I've seen situations with big textures where the opengl engine was slower than the raster engine.

There are two possible optimizations:
1. use the raw OpenGL call which updates an existing texture (glUpdateTex... something something) and render the texture in the item
2. divide one big surface into more smaller ones