Help in creating video streaming app

[superutsav]

Hi,

I am trying to create an application in qt to view a video stream from a webcamera. I'm using a Logitech webcam on a Mandrake 10 machine. I am able to get the video from the webcam into a buffer...
However, once i get the buffer and nextFrame() functions ready, i want to know how to display the video stream using qt.
I think openGL is the way to go, but i am not familiar with its usage, or how to use QGLWidget.
Also, the other forum posts say that using QPixmap is very slow, adn not useful for my project.

Please send me any links/code that will help me in this regard.

Thanks in advance
superutsav

[axeljaeger]

You can draw either a QPixmap or a QImage using a painter on a QGLWidget. Qt4 takes care of uploading the image as texture to the graphicscard and draw it using two triangles I think.

[superutsav]

Hi,

I should also mention what i'm using right now, which i'm sure is really horrible, inefficient code, and which keeps the processor busy in a loop, hence making it impossible for me to close the app without killing the process:

QPixmap myPixmap;
QImage image;
QLabel myLabel;
QWidget window;
QHBoxLayout layout;
layout.addWidget(myLabel);
window.setLayout(&layout);
 
while(1)
{
 image = QImage::QImage((uchar*)deviceAccessTool.nextFrame(), width,  height, QImage::Format_Rgb32);
image = myprocessingcode(image);
myPixmap = myPixmap.fromImage(image);
myLabel.setPixmap(myPixmap);
window.repaint();
}

QPixmap myPixmap;
QImage image;
QLabel myLabel;
QWidget window;
QHBoxLayout layout;
layout.addWidget(myLabel);
window.setLayout(&layout);

while(1)
{
 image = QImage::QImage((uchar*)deviceAccessTool.nextFrame(), width,  height, QImage::Format_Rgb32);
image = myprocessingcode(image);
myPixmap = myPixmap.fromImage(image);
myLabel.setPixmap(myPixmap);
window.repaint();
}

To copy to clipboard, switch view to plain text mode 

this code displays the streaming video without any flicker, yet. However, if i try to process the image before displaying, then it flikers a lot.

any code u guys can suggest would be of great help

Thanks in advance
superutsav

[high_flyer]

I would change your code along the lines of:

#
image =  QImage( width,  height, QImage::Format_Rgb32);
unsigned char *buff = image.bits();
 
while(1)
{
 memcpy(buff,(uchar*)deviceAccessTool.nextFrame(),/*size in bytes of the frame*/); //this should be faster then allocating each time a new image
image = myprocessingcode(image); //if you want this to fit in 25hz, then you should take in consideration that grabbing and processing should not be longer then 40ms!
myPixmap = myPixmap.fromImage(image);
myLabel.setPixmap(myPixmap);
window.repaint();
}

#
image =  QImage( width,  height, QImage::Format_Rgb32);
unsigned char *buff = image.bits();

while(1)
{
 memcpy(buff,(uchar*)deviceAccessTool.nextFrame(),/*size in bytes of the frame*/); //this should be faster then allocating each time a new image
image = myprocessingcode(image); //if you want this to fit in 25hz, then you should take in consideration that grabbing and processing should not be longer then 40ms!
myPixmap = myPixmap.fromImage(image);
myLabel.setPixmap(myPixmap);
window.repaint();
}

To copy to clipboard, switch view to plain text mode 

Also since I dont have much experiance with qt4, I would suggest to try (and can't say this IS better) the following:
instead of :

myPixmap = myPixmap.fromImage(image);
myLabel.setPixmap(myPixmap);

myPixmap = myPixmap.fromImage(image);
myLabel.setPixmap(myPixmap);

To copy to clipboard, switch view to plain text mode 

I would try to use the QPainter with a QImage as paint device (one of the new Qt4 fetures, it could be faster then setPixmap()):

//somewhere declared: QPainter painter(&myLabel);
painter.drawImage(maLabel.rect(),image,image.rect());

//somewhere declared: QPainter painter(&myLabel);
painter.drawImage(maLabel.rect(),image,image.rect());

To copy to clipboard, switch view to plain text mode 

So put togeather it would look somthing like:

image =  QImage( width,  height, QImage::Format_Rgb32);
unsigned char *buff = image.bits();
QPainter painter(&myLabel);
 
while(1)
{
 memcpy(buff,(uchar*)deviceAccessTool.nextFrame(),/*size in bytes of the frame*/);
image = myprocessingcode(image); //if you want this to fit in 25hz, then you should take in consideration that grabbing and processing should not be longer then 40ms!
painter.drawImage(maLabel.rect(),image,image.rect());
}

image =  QImage( width,  height, QImage::Format_Rgb32);
unsigned char *buff = image.bits();
QPainter painter(&myLabel);

while(1)
{
 memcpy(buff,(uchar*)deviceAccessTool.nextFrame(),/*size in bytes of the frame*/);
image = myprocessingcode(image); //if you want this to fit in 25hz, then you should take in consideration that grabbing and processing should not be longer then 40ms!
painter.drawImage(maLabel.rect(),image,image.rect());
}

To copy to clipboard, switch view to plain text mode 

Please note, under Qt3 I can show you how to do a video stream in a crossplatform way, but under Qt4, this code is only a suggestion I think you shoud try, and take in consideration I didn't test it!
Since drawing in Qt4 is done only in a paint event, this all needs to be in a paint event...

Any comments from you Qt4 experts?

[kroenecker]

So, out of curiosity, if I wanted to make an application that streams video content in QT3, how do I start? (ie what classes would be appropriate etc)

[superutsav]

memcpy(buff,(uchar*)deviceAccessTool.nextFrame(),/*size in bytes of the frame*/); //this should be faster then allocating each time a new image

memcpy(buff,(uchar*)deviceAccessTool.nextFrame(),/*size in bytes of the frame*/); //this should be faster then allocating each time a new image

To copy to clipboard, switch view to plain text mode 

I tried this... it doesnt speed up the frame rate at all... atleast not to the naked eye...

Also since I dont have much experiance with qt4, I would suggest to try (and can't say this IS better) the following:
instead of :

1.
      myPixmap = myPixmap.fromImage(image);
   2.
      myLabel.setPixmap(myPixmap);

1.
      myPixmap = myPixmap.fromImage(image);
   2.
      myLabel.setPixmap(myPixmap);

To copy to clipboard, switch view to plain text mode 

I would try to use the QPainter with a QImage as paint device (one of the new Qt4 fetures, it could be faster then setPixmap()):

1.
      //somewhere declared: QPainter painter(&myLabel);
   2.
      painter.drawImage(maLabel.rect(),image,image.rect());

1.
      //somewhere declared: QPainter painter(&myLabel);
   2.
      painter.drawImage(maLabel.rect(),image,image.rect());

To copy to clipboard, switch view to plain text mode 

I tried this, too... didn't speed up the thing much, either, but there is a small improvement.... i had some trouble getting it to work, though.

image = myprocessingcode(image); //if you want this to fit in 25hz, then you should take in consideration that grabbing and processing should not be longer then 40ms!

image = myprocessingcode(image); //if you want this to fit in 25hz, then you should take in consideration that grabbing and processing should not be longer then 40ms!

To copy to clipboard, switch view to plain text mode 

How do i measure the amount of time its taking to process?... do i have a way to do that?


Thanks
superutsav

[Mike]

well if you want to measure time, you could do it certainly like this:

QTime t;
t.start();
some_lengthy_task();
qDebug("Time elapsed: %d ms", t.elapsed());

QTime t;
t.start();
some_lengthy_task();
qDebug("Time elapsed: %d ms", t.elapsed());

To copy to clipboard, switch view to plain text mode 

[cloose]

I should also mention what i'm using right now, which i'm sure is really horrible, inefficient code, and which keeps the processor busy in a loop, hence making it impossible for me to close the app without killing the process:

You have to kill the process because of the "while(1) {}" infinite loop. You have to give Qt a chance to handle events. So for testing I would add a qapp->processEvents() call to the loop.

QPixmap myPixmap;
QImage image;
QLabel myLabel;
QWidget window;
QHBoxLayout layout;
layout.addWidget(myLabel);
window.setLayout(&layout);
 
while(1)
{
 image = QImage::QImage((uchar*)deviceAccessTool.nextFrame(), width,  height, QImage::Format_Rgb32);
image = myprocessingcode(image);
myPixmap = myPixmap.fromImage(image);
myLabel.setPixmap(myPixmap);
window.repaint();
}

QPixmap myPixmap;
QImage image;
QLabel myLabel;
QWidget window;
QHBoxLayout layout;
layout.addWidget(myLabel);
window.setLayout(&layout);

while(1)
{
 image = QImage::QImage((uchar*)deviceAccessTool.nextFrame(), width,  height, QImage::Format_Rgb32);
image = myprocessingcode(image);
myPixmap = myPixmap.fromImage(image);
myLabel.setPixmap(myPixmap);
window.repaint();
}

To copy to clipboard, switch view to plain text mode 

My idea would be to subclass QWidget and override the paintEvent() method.

following code untested!!

void VideoWidget::paintEvent()
{
    QPainter p( this );
 
    QImage image((uchar*)deviceAccessTool.nextFrame(), width,  height, QImage::Format_Rgb32);
    myprocessingcode(&image);
 
    p.drawImage (image.rect(), image);
}

void VideoWidget::paintEvent()
{
    QPainter p( this );

    QImage image((uchar*)deviceAccessTool.nextFrame(), width,  height, QImage::Format_Rgb32);
    myprocessingcode(&image);

    p.drawImage (image.rect(), image);
}

To copy to clipboard, switch view to plain text mode 

and somewhere else maybe....

{
    deviceAccessTool.loadNextFrame();
    connect(deviceAccessTool, SIGNAL(nextFrameAvailable()),
                videoWidget, SLOT(update()));
}

{
    deviceAccessTool.loadNextFrame();
    connect(deviceAccessTool, SIGNAL(nextFrameAvailable()),
                videoWidget, SLOT(update()));
}

To copy to clipboard, switch view to plain text mode 

[high_flyer]

So, out of curiosity, if I wanted to make an application that streams video content in QT3, how do I start? (ie what classes would be appropriate etc)

Nothing easier, here is an example of my QVideo plugin for 32 bit rgb frames:

/** No descriptions */
void QVideo::showRGB32(unsigned int *buff){
 int buffSize = width()*height()*sizeof(unsigned int);
 
 if(!m_image32)
  m_image32 =  new QImage(width(),height(),32);
 if(m_image32->width() != width() || m_image32->height() != height())
 {
  delete m_image32;
  m_image32 =  new QImage(width(),height(),32);
 } 
 
 memcpy(m_image32->bits(),buff,buffSize);
 if(!m_image32->isNull())
  bitBlt(this,0,0,m_image32);
// m_lastImage = m_image32; //<--this is not thread safe
 
}

/** No descriptions */
void QVideo::showRGB32(unsigned int *buff){
 int buffSize = width()*height()*sizeof(unsigned int);

 if(!m_image32)
  m_image32 =  new QImage(width(),height(),32);
 if(m_image32->width() != width() || m_image32->height() != height())
 {
  delete m_image32;
  m_image32 =  new QImage(width(),height(),32);
 } 

 memcpy(m_image32->bits(),buff,buffSize);
 if(!m_image32->isNull())
  bitBlt(this,0,0,m_image32);
// m_lastImage = m_image32; //<--this is not thread safe

}

To copy to clipboard, switch view to plain text mode 

[superutsav]

well if you want to measure time, you could do it certainly like this:

QTime t;
t.start();
some_lengthy_task();
qDebug("Time elapsed: %d ms", t.elapsed());

QTime t;
t.start();
some_lengthy_task();
qDebug("Time elapsed: %d ms", t.elapsed());

To copy to clipboard, switch view to plain text mode 

Hey... I tried this... and thank god i did....
Apparently the QPixmap::fromImage() function is taking an average of 33ms to run!!! Theres no way i can fit the whole thing into 40ms for a 25fps stream if just the fromImage is taking so long..... what do i do about this???

superutsav

[high_flyer]

Try to time the code I suggested before with teh painter.drawImage(), and see how long does that take...
You can also try to bitBlt() as I showed here (with Qt3), though bitBlt must do a conversion to pixmap internally, it works quite fast.
I have no problem using that for a sterao video stream each at 640x480 at 25hz.

[superutsav]

You have to kill the process because of the "while(1) {}" infinite loop. You have to give Qt a chance to handle events. So for testing I would add a qapp->processEvents() call to the loop.

Hey... Thanks... this processEvents() thing works great... i dont have to kill the process amymore.

You can also try to bitBlt() as I showed here (with Qt3), though bitBlt must do a conversion to pixmap internally, it works quite fast.

Does Qt4 have bitBlt??? It doesn't show in the documentation... only some vague stuff... but i'll try it nevertheless and tell u the result

Thanks
superutsav

[high_flyer]

Does Qt4 have bitBlt??? It doesn't show in the documentation... only some vague stuff... but i'll try it nevertheless and tell u the result

I would be surprozed if it did, since under Qt4 you can only draw in a paintEvent, and bitBlt() draws directly on a paint device when ever its called, which would break the Qt4 drawing rules.
But on the other hand, Qt4 added the ability for a painter to draw an image on the paint device which is basically what bitBlt does (well sorts of), and that is what i tried to do with the code i offered you at the beginning.

[superutsav]

Hi

I dont think Qt4 has bitBlt, after all... if i call bitBlt anywhere, it gives the following compilation error:

error: `bitBlt' undeclared (first use this function)

error: `bitBlt' undeclared (first use this function)

To copy to clipboard, switch view to plain text mode 

is there some other way of calling it other than directly saying bitBlt(...) ?? Which class does it belong to???

superutsav

[high_flyer]

Did you read my last post at all??
I answered all your last questions there.

[sabeesh]

hi,
Can you explain this line
(uchar*)deviceAccessTool.nextFrame(), width, height, QImage::Format_Rgb32)
from you code. in this , " deviceAccessTool.nextFrame() " what you mean with this?

[SimbadSubZero]

hi superutsav,
I am a new at QT video stream.
Could please explain the mention of myprocessingcode(image) function. Does it the buffer which come from QIODevice(function read)?

Thanks,
Simbad.