PDA

View Full Version : Processing and Displaying Image on Widget



mhb88
28th March 2016, 08:54
Hello everyone,

It's been a while that I am trying to come up with a method to do some image processing and displaying results in a Widget, that is a child of MainWindow, on frames that I get from a camera (maybe at rate of 25 frames/sec). So, I started by reading tones of documents and forums and etc to find the method, and so far I have been able to implement this application. Since image processing algorithms are computationally intensive, I thought it would be a good idea to use QThread in my project. I found mandelbrot example extremely useful for my purpose.
http://doc.qt.io/qt-5/qtcore-threads-mandelbrot-example.html

Like the example:

I created a class to display image (mQtPaintWidget). Then I added a Widget to my central widget in mainwindow.ui and promoted to this class and called it Screen. I also added a Text Edit to my central widget and called it Console.
I created a class to assign a thread for my image processing algorithms (RenderThread).



Before writing a class to handle my camera, I wanted to test if I can display noise using qrand() function but it does not run the way I expect. I got it to work till the paintevent checks the pixmap to see if it is NULL and draws a text on Screen.
I copied my codes down here. I would appreciate if any body can suggest any solution.

cheers :)



main.cpp

#include "mainwindow.h"
#include <QApplication>

int main(int argc, char *argv[])
{
QApplication app(argc, argv);
MainWindow w;
w.show();

return app.exec();
}

mainwindow.h

#ifndef MAINWINDOW_H
#define MAINWINDOW_H

#include <QMainWindow>
#include <QWidget>

namespace Ui {
class MainWindow;
}

class MainWindow : public QMainWindow
{
Q_OBJECT

public:
explicit MainWindow(QWidget *parent = 0);
~MainWindow();

private:
Ui::MainWindow *ui;
};

#endif // MAINWINDOW_H

mainwindow.cpp

#include "mainwindow.h"
#include "ui_mainwindow.h"

MainWindow::MainWindow(QWidget *parent) :
QMainWindow(parent),
ui(new Ui::MainWindow)
{
ui->setupUi(this);

QPalette p = palette();
p.setColor(QPalette::Base, Qt::black);
p.setColor(QPalette::Text, Qt::green);
ui->Console->setPalette(p);
}

MainWindow::~MainWindow()
{
delete ui;
}

renderthread.h

#ifndef RENDERTHREAD_H
#define RENDERTHREAD_H

#include <QWidget>
#include <QMutex>
#include <QThread>
#include <QWaitCondition>

QT_BEGIN_NAMESPACE
class QImage;
QT_END_NAMESPACE

class RenderThread : public QThread
{
Q_OBJECT

public:
RenderThread(QObject *parent = 0);
~RenderThread();

void render();

signals:
void renderedImage(const QImage &image);

protected:
void run() Q_DECL_OVERRIDE;

private:
QMutex mutex;
QWaitCondition condition;
QSize frameSize;
bool restart;
bool abort;
};

#endif // RENDERTHREAD_H

renderthread.cpp

#include "renderthread.h"

RenderThread::RenderThread(QObject *parent)
: QThread(parent)
{
restart = false;
abort = false;
}

RenderThread::~RenderThread()
{
mutex.lock();
abort = true;
condition.wakeOne();
mutex.unlock();

wait();
}

void RenderThread::render()
{
QMutexLocker locker(&mutex);

if (!isRunning()) {
start(LowPriority);
} else {
restart = true;
condition.wakeOne();
}
}

void RenderThread::run()
{
forever
{
mutex.lock();
if (restart || abort)
return;
mutex.unlock();

uint imWidth = 600;
uint imHeight = 480;
QSize frameSize(imWidth,imHeight);
QImage image(frameSize,QImage::Format_Indexed8);

for(uint i = 0; i < imHeight; ++i)
{
uint *scanLine = reinterpret_cast<uint *>(image.scanLine(i));
for(uint j = 0; j < imWidth; ++j)
*scanLine++ = (qrand() > RAND_MAX/2) * 255;
}

if (!restart)
emit renderedImage(image);

mutex.lock();
if (!restart)
condition.wait(&mutex);
restart = false;
mutex.unlock();
}

}

mqtpaintwidget.h

#ifndef MQTPAINTERWIDGET_H
#define MQTPAINTERWIDGET_H

#include <QWidget>
#include <QPixmap>
#include <QPainter>
#include "renderthread.h"

class mQtPainterWidget : public QWidget
{
Q_OBJECT
public:
explicit mQtPainterWidget(QWidget *parent = 0);

protected:
void paintEvent(QPaintEvent *event) Q_DECL_OVERRIDE;

private slots:
void updatePixmap(const QImage &image);

private:
RenderThread thread;
QPixmap pixmap;
};

#endif // MQTPAINTERWIDGET_H

mqtpaintwidget.cpp

#include "mqtpainterwidget.h"

mQtPainterWidget::mQtPainterWidget(QWidget *parent) :
QWidget(parent)
{
connect(&thread, SIGNAL(renderedImage(QImage)), this, SLOT(updatePixmap(QImage)));
//thread.render();
}

void mQtPainterWidget::paintEvent(QPaintEvent *event)
{
Q_UNUSED(event);

QPainter painter(this);
painter.fillRect(rect(), Qt::black);

if (pixmap.isNull()) {
painter.setPen(Qt::green);
painter.drawText(rect(), Qt::AlignCenter, tr("Rendering initial image, please wait..."));
return;
}

painter.drawPixmap(0,0,pixmap);
}

void mQtPainterWidget::updatePixmap(const QImage &image)
{
pixmap = QPixmap::fromImage(image);
update();
}

11830

anda_skoa
28th March 2016, 10:21
Well, your paintEvent() make the widget display text if the pixmap is null, which is the default state if the "pixmap" member.
Since you haven't started the thread, it couldn't have sent an image yet, so the "pixmap" will stay null.

Btw, your early exit in run() leaves the mutex locked and you are accessing "restart" once without proper locking.

Cheers,
_

mhb88
28th March 2016, 17:39
Thanks for your quick response :)

I have a problem with starting the thread. start function is in my render() routine in my RenderThread class. I have tried to call render routine from various locations in my code, but all of them resulted in crashing right at the startup of my application.

I even added a timer event slot to my RenderThread class and call it when I construct the object of this class but got the same result, crashing about a second after the startup!
I have no idea why my app crashes like this!
you can see my code down here.



private slots:
void timerEvent(QTimerEvent *event);



RenderThread::RenderThread(QObject *parent)
: QThread(parent)
{
restart = false;
abort = false;

startTimer(500);
}



void RenderThread::timerEvent(QTimerEvent *event)
{
Q_UNUSED(event);

render();
}

d_stranz
28th March 2016, 17:59
You are creating your QImage on the stack in your render thread, then passing it as a reference to your main window via a signal. Perhaps the crash is because the QImage has gone out of scope by the time the slot receives the reference. Try making the QImage either a member variable of the thread class or passing it by value (so a copy gets made).

If you make it a member variable, then you might have concurrency problems if the main window is trying to read the image at the same time the render thread is changing it.

anda_skoa
28th March 2016, 18:35
You are creating your QImage on the stack in your render thread, then passing it as a reference to your main window via a signal. Perhaps the crash is because the QImage has gone out of scope by the time the slot receives the reference. Try making the QImage either a member variable of the thread class or passing it by value (so a copy gets made).

No, I think that's ok.
The image copied into a QVariant which is then put into an event and sent to the receiver object's event loop.
While QImage is implicitly shared the "going out of scope" should make the copy inside the variant the only surviving reference and thus safe to access from the main thread.


I have a problem with starting the thread. start function is in my render() routine in my RenderThread class. I have tried to call render routine from various locations in my code, but all of them resulted in crashing right at the startup of my application.

Then you'll have to debug why it crashes.
Not running the image source will not give you any images.

My guess is that you are accessing memory areas that are not valid while looping over the image.
Your inner loop's step size is 1 uint per loop, an uint is usually 4 bytes.
But your image format is one byte per pixel.

Cheers,
_

mhb88
28th March 2016, 19:06
My guess is that you are accessing memory areas that are not valid while looping over the image.
Your inner loop's step size is 1 uint per loop, an uint is usually 4 bytes.
But your image format is one byte per pixel.


oh, Wow!! I can't believe that I let this warning confuse me!!


uchar *QImage::scanLine(int i)
Returns a pointer to the pixel data at the scanline with index i. The first scanline is at index 0.
The scanline data is aligned on a 32-bit boundary.
Warning: If you are accessing 32-bpp image data, cast the returned pointer to QRgb* (QRgb has a 32-bit size) and use it to read/write the pixel value. You cannot use the uchar* pointer directly, because the pixel format depends on the byte order on the underlying platform. Use qRed(), qGreen(), qBlue(), and qAlpha() to access the pixels.

thank you very much anda_skoa

cheers :)

mhb88
29th March 2016, 04:16
as you see in my code I posted at the beginning of this thread, I have bunch of classes already in my project and I am going to add a few more as I progress in my project. I came up with the idea to add a TextEdit to my mainwindow.ui and called it Console to display some text there through out my code.
I want to be able to do this from all my classes.
At first I thought there is no easier task than this, but apparently I can't find a solution.
would you guys help me on this?

cheers :)

anda_skoa
29th March 2016, 09:12
You have different options:

- add signals that emit new "console" text to classes that need it and connect to the text edit or a slot in main window
- pass the pointer to the text edit or main window to wherever this is needed
- make the pointer to the text edit or the main window globally accessible

Cheers,
_

mhb88
31st March 2016, 03:32
OK, I got everything working as the base of my project. At this point I can communicate with camera, and receive its frames, and just display them in my widget.
But I have an issue.
My camera supports up to 100 frames/sec, but I am barely displaying 15 frames/sec.
My camera resolution is 480x640 and I am just receiving 8bits grayscale frames. So each frame has 480 x 640 x 8 = 300KBytes of data and if I want to receive them at rate of 100 fps, I have to receive them 300KBytes/frame x 100 frames/sec = 30000 KBytes/sec, about 30MBytes/sec, which is half of the speed of the USB2.0. So I think my USB2.0 should be able to do handle that amount of data transfer.

So I think the issue is in my software, but I am not doing any processing in my code, I am just displaying frames. I have created a class with a paint event, and I have promoted a Qwidget to this class to display on it.

I don't know if Qwidget can handle transferring this amount of data?
Does any one has any suggestion or solution?

cheers :)

anda_skoa
31st March 2016, 08:56
For such high througput rendering you probably have to use a more direct rendering approach, e.g. OpenGL.

It might be worthwhile to look into creating a QCamera backend and then use the existing video widget infrastructure.

Cheers,
_

mhb88
31st March 2016, 17:29
Do you know any tutorials that start from foundation and builds up to it?
I don't have any experience with OpenGL.
I found most tutorials to be about QGLWidget.
But in the new versions of Qt 5, QOpenGLWidget is intended to be a modern replacement for QGLWidget.
I haven't been able find a good tutorial for QOpenGLWidget.

cheers

anda_skoa
31st March 2016, 17:48
No, sorry, I haven't had anything to do with that yet myself either.

Cheers,
_

mhb88
15th April 2016, 04:17
I was wondering if this would work?
a function is getting a pointer to a QImage as a parameter.
I want to copy the entire QImage to a buffer of that is going to hold 100 QImages.



public:
void function(QImage *frame);

protected:
enum{size = 100};
QImage *data[size];
int front; // points to the front of the buffer




void Buffer::function(QImage *frame)
{
++front; // Increment front.
if (front == size)
front = 0; // Wrap around.

data[front] = frame;
}


cheers :)

anda_skoa
15th April 2016, 10:24
Sure, why not.
I assume you handle deletion of the QImage pointers somewhere else?

Cheers,
_