PDA

View Full Version : Replace Pixmap in GraphicsScene



Royce
5th November 2016, 19:46
I'd really like to attach to my web camera, process the frames I get out of it, and then display the processed frame in a window with some super-imposed markings.

The strategy I've been trying is to have a QGraphicsView with a QGraphicsScene that has a QGraphicsPixmap added. I then use a custom VideoSurface to grab frames. Right now I'm trying to just transfer the frames to the QGraphicsView (no extra markings or processing) but if I call setPixmap the program will crash just after the frame handler returns. If I skip the call to setPixmap, it doesn't crash. Could anyone point me in the right direction?


MainWindow::MainWindow(QWidget *parent) :
QMainWindow(parent),
ui(new Ui::MainWindow),
frame(640, 480)
{
ui->setupUi(this);
QList<QCameraInfo> cameras = QCameraInfo::availableCameras();
cam = new QCamera(cameras.first(), this);
cam->load();
frame_grabber = new CameraFrameGrabber();
cam->setViewfinder(frame_grabber);
cam->setCaptureMode(QCamera::CaptureViewfinder);
connect(frame_grabber, SIGNAL(frameAvailable(QImage)), this, SLOT(handleFrame(QImage)));
cam->start();
frame_timer.start();
scene = new QGraphicsScene();
pix = new QGraphicsPixmapItem(frame);
scene->addItem(pix);
ui->gv_viewfinder->setScene(scene);
}

MainWindow::~MainWindow()
{
delete ui;
}

void MainWindow::handleFrame(const QImage& img)
{
frame_times.push_back(frame_timer.elapsed());
frame_timer.restart();
if (frame_times.size() > 10) frame_times.pop_front();
int frame_count = frame_times.size();
int frame_total = 0;
for(int i = 0; i < frame_count; ++i)
{
frame_total += frame_times[i];
}
double fps = 0;
if (frame_count)
{
fps = (frame_total / frame_count);
}
ui->sb_fps->setValue(fps);

bool good_pix = frame.convertFromImage(img);
if (good_pix) pix->setPixmap(frame);
this->update();
}


bool CameraFrameGrabber::present(const QVideoFrame &frame)
{
if (frame.isValid()) {
QVideoFrame cloneFrame(frame);
cloneFrame.map(QAbstractVideoBuffer::ReadOnly);
const QImage image(cloneFrame.bits(),
cloneFrame.width(),
cloneFrame.height(),
QVideoFrame::imageFormatFromPixelFormat(cloneFrame .pixelFormat()));
emit frameAvailable(image);
cloneFrame.unmap();
return true;
}
return false;
}

anda_skoa
6th November 2016, 13:44
The observed behavior would suggest that the "pix" pointer is not valid, thus accessing it crashes.

Can you try initializing it before starting the camera?

Cheers,
_

Royce
6th November 2016, 14:43
That was a good thought! But it doesn't seem to be the issue. I've also tried a slightly different approach in the handleFrame method, but it doesn't seem to make a difference either.

The thing is that it doesn't actually crash in handleFrame. I guess it crashes somewhere in the event handler. Certainly it is a null pointer, but the null pointer is way down the stack during a memcpy operation.

I've included the stack trace as well.


The current code:

MainWindow::MainWindow(QWidget *parent) :
QMainWindow(parent),
ui(new Ui::MainWindow),
frame(640, 480)
{
ui->setupUi(this);
scene = new QGraphicsScene();
pix = new QGraphicsPixmapItem(frame);
scene->addItem(pix);
ui->gv_viewfinder->setScene(scene);
QList<QCameraInfo> cameras = QCameraInfo::availableCameras();
cam = new QCamera(cameras.first(), this);
cam->load();
frame_grabber = new CameraFrameGrabber();
cam->setViewfinder(frame_grabber);
cam->setCaptureMode(QCamera::CaptureViewfinder);
connect(frame_grabber, SIGNAL(frameAvailable(QImage)), this, SLOT(handleFrame(QImage)));
cam->start();
frame_timer.start();

}

MainWindow::~MainWindow()
{
delete ui;
}

void MainWindow::handleFrame(const QImage& img)
{
frame_times.push_back(frame_timer.elapsed());
frame_timer.restart();
if (frame_times.size() > 10) frame_times.pop_front();
int frame_count = frame_times.size();
int frame_total = 0;
for(int i = 0; i < frame_count; ++i)
{
frame_total += frame_times[i];
}
double fps = 0;
if (frame_count)
{
fps = (frame_total / frame_count);
}
ui->sb_fps->setValue(fps);

scene->removeItem(pix);
delete pix;
frame = QPixmap::fromImage(img);
pix = scene->addPixmap(frame);
scene->invalidate();

this->update();
}


1 memcpy VCRUNTIME140D 0x7ffc229b1e10
2 qt_blend_rgb32_on_rgb32 qblendfunctions.cpp 399 0x7ffbfbf7bb85
3 qt_blend_rgb32_on_rgb32_sse2 qdrawhelper_sse2.cpp 139 0x7ffbfc32c17d
4 QRasterPaintEnginePrivate::drawImage qpaintengine_raster.cpp 1021 0x7ffbfc03b9bd
5 QRasterPaintEngine::drawImage qpaintengine_raster.cpp 2133 0x7ffbfc03235b
6 QRasterPaintEngine::drawPixmap qpaintengine_raster.cpp 2019 0x7ffbfc03163b
7 QPainter::drawPixmap qpainter.cpp 5070 0x7ffbfc05799c
8 QGraphicsPixmapItem::paint qgraphicsitem.cpp 9769 0x5de350bf
9 QGraphicsScenePrivate::draw qgraphicsscene.cpp 4961 0x5de7fe17
10 QGraphicsScenePrivate::drawSubtreeRecursive qgraphicsscene.cpp 4855 0x5de7f73b
11 QGraphicsScenePrivate::drawItems qgraphicsscene.cpp 4714 0x5de7e9a0
12 QGraphicsView::paintEvent qgraphicsview.cpp 3551 0x5dec8c82
13 QWidget::event qwidget.cpp 8931 0x5d910fb1
14 QFrame::event qframe.cpp 550 0x5db0c62f
15 QAbstractScrollArea::viewportEvent qabstractscrollarea.cpp 1213 0x5dbfe81c
16 QGraphicsView::viewportEvent qgraphicsview.cpp 2974 0x5dec66a4
17 QAbstractScrollAreaPrivate::viewportEvent qabstractscrollarea_p.h 111 0x5d899c55
18 QAbstractScrollAreaFilter::eventFilter qabstractscrollarea_p.h 127 0x5dc0375d
19 QCoreApplicationPrivate::sendThroughObjectEventFil ters qcoreapplication.cpp 1099 0x5d0f8644
20 QApplicationPrivate::notify_helper qapplication.cpp 3795 0x5d8a5e6e
21 QApplication::notify qapplication.cpp 3762 0x5d8a34c9
22 QCoreApplication::notifyInternal2 qcoreapplication.cpp 988 0x5d0f78c6
23 QCoreApplication::sendSpontaneousEvent qcoreapplication.h 234 0x5d264c3b
24 QWidgetPrivate::sendPaintEvent qwidget.cpp 5699 0x5d91af5d
25 QWidgetPrivate::drawWidget qwidget.cpp 5640 0x5d91aa57
26 QWidgetBackingStore::doSync qwidgetbackingstore.cpp 1357 0x5d8bf802
27 QWidgetBackingStore::sync qwidgetbackingstore.cpp 1152 0x5d8bc8a3
28 QWidgetPrivate::syncBackingStore qwidget.cpp 1957 0x5d91d650
29 QWidget::event qwidget.cpp 9084 0x5d911617
30 QMainWindow::event qmainwindow.cpp 1544 0x5db380b1
31 QApplicationPrivate::notify_helper qapplication.cpp 3799 0x5d8a5e8e
32 QApplication::notify qapplication.cpp 3762 0x5d8a34c9
33 QCoreApplication::notifyInternal2 qcoreapplication.cpp 988 0x5d0f78c6
34 QCoreApplication::sendEvent qcoreapplication.h 231 0x5d1027f2
35 QCoreApplicationPrivate::sendPostedEvents qcoreapplication.cpp 1649 0x5d0f9112
36 QCoreApplication::sendPostedEvents qcoreapplication.cpp 1504 0x5d0f5c4c
37 QGraphicsViewPrivate::dispatchPendingUpdateRequest s qgraphicsview_p.h 204 0x5d88ec31
38 QGraphicsScenePrivate::_q_emitUpdated qgraphicsscene.cpp 383 0x5de7751b
39 QGraphicsScene::qt_static_metacall moc_qgraphicsscene.cpp 180 0x5de6e856
40 QMetaCallEvent::placeMetaCall qobject.cpp 503 0x5d156c81
41 QObject::event qobject.cpp 1263 0x5d14f6df
42 QGraphicsScene::event qgraphicsscene.cpp 3522 0x5de74608
43 QApplicationPrivate::notify_helper qapplication.cpp 3799 0x5d8a5e8e
44 QApplication::notify qapplication.cpp 3159 0x5d8a0713
45 QCoreApplication::notifyInternal2 qcoreapplication.cpp 988 0x5d0f78c6
46 QCoreApplication::sendEvent qcoreapplication.h 231 0x5d1027f2
47 QCoreApplicationPrivate::sendPostedEvents qcoreapplication.cpp 1649 0x5d0f9112
48 QEventDispatcherWin32::sendPostedEvents qeventdispatcher_win.cpp 1295 0x5d1b180a
49 QWindowsGuiEventDispatcher::sendPostedEvents qwindowsguieventdispatcher.cpp 82 0x7ffbfddfc204
50 qt_internal_proc qeventdispatcher_win.cpp 445 0x5d1af521
51 CallWindowProcW USER32 0x7ffc29f41c24
52 DispatchMessageW USER32 0x7ffc29f4156c
53 QEventDispatcherWin32::processEvents qeventdispatcher_win.cpp 845 0x5d1aff2b
54 QWindowsGuiEventDispatcher::processEvents qwindowsguieventdispatcher.cpp 74 0x7ffbfddfc1c4
55 QEventLoop::processEvents qeventloop.cpp 135 0x5d0f24e8
56 QEventLoop::exec qeventloop.cpp 210 0x5d0f272e
57 QCoreApplication::exec qcoreapplication.cpp 1261 0x5d0f55ff
58 QGuiApplication::exec qguiapplication.cpp 1640 0x7ffbfbc63208
59 QApplication::exec qapplication.cpp 2976 0x5d8a021a
60 main main.cpp 10 0x7ff6596b2759
61 WinMain qtmain_win.cpp 123 0x7ff6596bb14d
62 invoke_main exe_common.inl 109 0x7ff6596b8d7d
63 __scrt_common_main_seh exe_common.inl 264 0x7ff6596b8c1e
64 __scrt_common_main exe_common.inl 309 0x7ff6596b8ade
65 WinMainCRTStartup exe_winmain.cpp 17 0x7ff6596b8d99
66 BaseThreadInitThunk KERNEL32 0x7ffc2a4a8364
67 RtlUserThreadStart ntdll 0x7ffc2c1a5e91

anda_skoa
6th November 2016, 16:53
Hmm, weird.

Have you tried saving the image? I.e. to see if it is somehow damaged?
Though I would assume that this would already cause a problem in QPixmap::fromImage().

Or maybe try showing the resulting pixmap in a separate QLabel, to verify it being OK.

Cheers,
_

Royce
8th November 2016, 00:22
Hmm, weird.

Have you tried saving the image? I.e. to see if it is somehow damaged?
Though I would assume that this would already cause a problem in QPixmap::fromImage().

Or maybe try showing the resulting pixmap in a separate QLabel, to verify it being OK.

Cheers,
_

Well setting to a label results in a crash in exactly the same place. Oddly, saving to a jpg file works fine (and not attempting to display onscreen). Its upside down, but that's is the only thing odd I see about the image.

I feel like there is something fundamental about what's wrong. Something I'm just not aware of in the Qt world. I don't think its threads, all the crashes and breakpoints show WinMain in the stack. But I have no idea what it could be.

Thanks for your help, anyway.

Royce
8th November 2016, 04:45
Perhaps it is something about the event handler. If I set the label to one of the stored pixmaps in the constructor, it works. I see the image. But the captured frame still crashes it.


MainWindow::MainWindow(QWidget *parent) :
QMainWindow(parent),
ui(new Ui::MainWindow),
frame(640, 480),
pix_no(0)
{
ui->setupUi(this);

frame.load("campix1.jpg");
ui->label->setPixmap(frame);

QList<QCameraInfo> cameras = QCameraInfo::availableCameras();
cam = new QCamera(cameras.first(), this);
cam->load();
frame_grabber = new CameraFrameGrabber();
cam->setViewfinder(frame_grabber);
cam->setCaptureMode(QCamera::CaptureViewfinder);
connect(frame_grabber, SIGNAL(frameAvailable(QImage)), this, SLOT(handleFrame(QImage)));
cam->start();
frame_timer.start();

}

MainWindow::~MainWindow()
{
delete ui;
}

void MainWindow::handleFrame(const QImage& img)
{
frame_times.push_back(frame_timer.elapsed());
frame_timer.restart();
if (frame_times.size() > 10) frame_times.pop_front();
int frame_count = frame_times.size();
int frame_total = 0;
for(int i = 0; i < frame_count; ++i)
{
frame_total += frame_times[i];
}
double fps = 0;
if (frame_count)
{
fps = (frame_total / frame_count);
}
ui->sb_fps->setValue(fps);

frame = QPixmap::fromImage(img);

QString fname("campix");
fname += QString::number(++pix_no);
fname += ".jpg";
bool success = frame.save(fname);

ui->label->setPixmap(frame);

this->update();
}

This is the stack while stopped at a breakpoint in the handleFrame event.


1 MainWindow::handleFrame mainwindow.cpp 62 0x7ff6d7082e44
2 MainWindow::qt_static_metacall moc_mainwindow.cpp 73 0x7ff6d7087bb5
3 QMetaObject::activate qobject.cpp 3742 0x5d14daf2
4 QMetaObject::activate qobject.cpp 3603 0x5d14d268
5 CameraFrameGrabber::frameAvailable moc_cameraframegrabber.cpp 128 0x7ff6d7087f1b
6 CameraFrameGrabber::present cameraframegrabber.cpp 56 0x7ff6d7086e00
7 DSCameraSession::presentFrame dscamerasession.cpp 683 0x7ffc074ca264
8 DSCameraSession::qt_static_metacall moc_dscamerasession.cpp 111 0x7ffc074da2e6
9 QMetaCallEvent::placeMetaCall qobject.cpp 503 0x5d156c81
10 QObject::event qobject.cpp 1263 0x5d14f6df
11 QApplicationPrivate::notify_helper qapplication.cpp 3799 0x5d8a5e8e
12 QApplication::notify qapplication.cpp 3159 0x5d8a0713
13 QCoreApplication::notifyInternal2 qcoreapplication.cpp 988 0x5d0f78c6
14 QCoreApplication::sendEvent qcoreapplication.h 231 0x5d1027f2
15 QCoreApplicationPrivate::sendPostedEvents qcoreapplication.cpp 1649 0x5d0f9112
16 QEventDispatcherWin32::sendPostedEvents qeventdispatcher_win.cpp 1295 0x5d1b180a
17 QWindowsGuiEventDispatcher::sendPostedEvents qwindowsguieventdispatcher.cpp 82 0x7ffbfc70c204
18 qt_internal_proc qeventdispatcher_win.cpp 445 0x5d1af521
19 CallWindowProcW USER32 0x7ffc29f41c24
20 DispatchMessageW USER32 0x7ffc29f4156c
21 QEventDispatcherWin32::processEvents qeventdispatcher_win.cpp 845 0x5d1aff2b
22 QWindowsGuiEventDispatcher::processEvents qwindowsguieventdispatcher.cpp 74 0x7ffbfc70c1c4
23 QEventLoop::processEvents qeventloop.cpp 135 0x5d0f24e8
24 QEventLoop::exec qeventloop.cpp 210 0x5d0f272e
25 QCoreApplication::exec qcoreapplication.cpp 1261 0x5d0f55ff
26 QGuiApplication::exec qguiapplication.cpp 1640 0x7ffbfccf3208
27 QApplication::exec qapplication.cpp 2976 0x5d8a021a
28 main main.cpp 10 0x7ff6d7082749
29 WinMain qtmain_win.cpp 123 0x7ff6d708afbd
30 invoke_main exe_common.inl 109 0x7ff6d7088bed
31 __scrt_common_main_seh exe_common.inl 264 0x7ff6d7088a8e
32 __scrt_common_main exe_common.inl 309 0x7ff6d708894e
33 WinMainCRTStartup exe_winmain.cpp 17 0x7ff6d7088c09
34 BaseThreadInitThunk KERNEL32 0x7ffc2a4a8364
35 RtlUserThreadStart ntdll 0x7ffc2c1a5e91

anda_skoa
8th November 2016, 18:17
Very strange, the conversion to QPixmap should have consumed the original pixel data, anything afterwards should be fine.

Can you try something like this?



QImage imgCopy = img;
imgCopy.detach();
QPixmap frame = QPixmap::fromImage(imgCopy);


Cheers,
_

Royce
8th November 2016, 21:18
Woah! That worked!

In my code the QPixmap frame is a member variable. So it is not falling out of scope. The QImage however, as a stack variable, is falling out of scope. So I guess we have to proceed from the supposition that a reference to the image was being held.

Reviewing the documentation I see that the overload:


QPixmap QPixmap::fromImage(QImage &&image, Qt::ImageConversionFlags flags = Qt::AutoColor)

says "...without copying if possible." I guess that actually applies to both overloads.

I suppose the lesson learned is to look for detach()? If I see it I need to be mindful of the possibility that there may be lingering references if I don't call it?

Anyway, thanks again. I can move forward with my project.

anda_skoa
8th November 2016, 23:04
Maybe but I expect more a threading problem.

Anyway, since it works,lets not dwell on that :)

Cheers,
_

Royce
9th November 2016, 06:47
Hmm. It'd have to be an implicit thread. I don't start one and all crashes and break points show WinMain in the stack trace.