How could I get the image buffer of QCamera?

[stereoMatching]

QByteArray device = QCamera::availableDevices()[0];
QCamera camera(device);
QVideoWidget surface;
surface.resize(320, 240);
camera.setViewfinder(&surface);

How could I get the image capture by QCamera?
I want to do some post-processing on the camera video
QCameraImageCapture is too slow for real time

Anything similar to following codes?



QCamera camera(device);
QImage img = camera.getQImage(); //take the image buffer of camera

[stampede]

You can subclass QAbstractVideoSurface and get the image data from the incoming QVideoFrame object in present method:


class MyVideoSurface: public QAbstractVideoSurface{
Q_OBJECT
public:
MyVideoSurface(QObject * parent=NULL) : QAbstractVideoSurface(parent)
{}

QList<QVideoFrame::PixelFormat>
supportedPixelFormats(QAbstractVideoBuffer::Handle Type type) const{
...
return QList<QVideoFrame::PixelFormat>() << QVideoFrame::Format_RGB24 << ...; // here return whatever formats you will handle
}

bool present(const QVideoFrame& frame){
if (frame.isValid() && ...) {
QVideoFrame cloneFrame(frame);
cloneFrame.map(QAbstractVideoBuffer::ReadOnly);
const QImage img(cloneFrame.bits(),
cloneFrame.width(),
cloneFrame.height(),
QVideoFrame::imageFormatFromPixelFormat(cloneFrame .pixelFormat()));

// do something with the image ...

cloneFrame.unmap();
return true;
}
return false;
}

};

Of course this way you cannot use QVideoWidget as the QCamera object can have only one viewfinder attached - so you'll have to handle the rendering yourself.

[stereoMatching]

Thanks, but this solution is a little bit verbose(no offense)
do we have an easier way to get the image?

besides, do we have to copy the frame before we can
do anything to it?

[stampede]

this solution is a little bit verbose(no offense)
None taken :) I agree that this is quite a lot of code just to get one QImage object.

besides, do we have to copy the frame before we can do anything to it?
You can access pixel data from QVideoFrame object only if its mapped - and since map() is non-const method, you cannot call it on a const object.
Anyway don't worry about the copy constructor, because copy constructor of QVideoFrame creates a shallow copy. When you call map() on the frame object, it may transfer the underlying pixel data to RAM (for example if frame data is stored in video memory), but I think it depends on the implementation - I can imagine for some implementations this pixel data can be already in the main memory. ReadOnly flag probably allows even more optimizations as the mapped pixel data will not be written back to original video frame data, but I'm not really an expert on QCamera / QVideo* stuff so you use my advice with caution :)
Furthermore, QImage constructor does not copy the image buffer, so you'll have to copy the image only when you process it.
Btw. what is your OS and qcamera backend ? I remember using QCamera with gstreamer backend on linux and to be honest I was not very happy with it - I couldn't get it to list possible framerates / resolutions (tested with few uvc cameras), let alone change the capture parameters...

[stereoMatching]

Btw. what is your OS and qcamera backend ?

The OS I intent to deploy are

OS X 10.8.5(mac mini)
win 7 64bits
android 4.1.2(Qt5.2 rc1 have bugs on this platform yet)

I use the default back end come with the binary
In truth, I don't know how to change the backend

At first, I would like to use the cv::VideoCapture to process the image
but I don't know how to link the camera lib of opencv on android with Qt5
so I decided to give QCamera a try

If I am correct, I have to process the image frame by frame in this present function?

[stampede]

Yes, you could do that

[hasti]

I 've implemented a camera like your code. I want to use QVideoFrame for displaying camera stream on android. I've converted each frame to QImage and then used QPaimter.
but frames are showed very slowly. here is my code:

bool MainWindow::present(const QVideoFrame &frame)
{

QVideoFrame cloneFrame(frame);

if(cloneFrame.map(QAbstractVideoBuffer::ReadOnly))
{
img = QImage(cloneFrame.size(), QImage::Format_ARGB32);
qt_convert_NV21_to_ARGB32(cloneFrame.bits(),
(quint32 *)img.bits(),
cloneFrame.width(),
cloneFrame.height());

widget->update();

cloneFrame.unmap();
}
return true;
}

....
void MainWindow::paintEvent(QPaintEvent *event)
{
QRectF target(10.0, 20.0, 600.0, 650.0);
QRectF source(0.0, 0.0, 600.0, 650.0);
QPainter painter(this);
painter.begin(&img);
painter.drawImage(target, img, source);
painter.end();

}

[liprandi]

You example give me a memory leak, if I use several time.
Any idea how to fix it?