Results 1 to 3 of 3

Thread: OpenCV image sending through Qt socket

  1. #1
    Join Date
    Nov 2007
    Location
    Italy
    Posts
    615
    Thanks
    22
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default OpenCV image sending through Qt socket

    Hi,
    I'm trying to port to QT, a program that sends webcam frames (cv :: Mat) through Qt socket, like a streaming video. I could do it successfully using C++ and pthreads only but I have issues using the QTcpSocket mechanism.

    The program logic is really simple: at client side i capture webcam frames and I send to the server after connecting.

    In a timer slot I capture the frames, resize to 640x480, convert to gray scale and write to a socket:

    Qt Code:
    1. // The main code of the timer slot
    2. QObject::connect(&captureLoopTimer, &QTimer::timeout, [&]()
    3. {
    4. mutex.lock();
    5. capture.read(frame);
    6. if(frame.empty())
    7. {
    8. qWarning() << "Empty frame";
    9. mutex.unlock();
    10. return;
    11. }
    12. else
    13. {
    14. qDebug() << "Frame received";
    15. }
    16. // I correctly got a webcam frame -> convert to grayscale
    17. flip(frame, frame, 1);
    18. cvtColor(frame, gray, CV_BGR2GRAY);
    19.  
    20. // Frame is ready to be written to a tcp socket
    21. isDataReady = 1;
    22. mutex.unlock();
    23. });
    To copy to clipboard, switch view to plain text mode 

    The frames are written to a QTcpSocket in an external thread launched using a QtConcurrent::run(writeFramesToSocket), following the main code

    Qt Code:
    1. void writeFramesToSocket()
    2. {
    3. QTcpSocket socket;
    4. socket.connectToHost("127.0.0.1", 2222);
    5. if(socket.waitForConnected(5000))
    6. {
    7. qDebug() << "Connected...";
    8. }
    9. else
    10. {
    11. qWarning() << "Can't connect to host";
    12. return;
    13. }
    14.  
    15. reshaped = (gray.reshape(0, 1)); // To make it continuous
    16. unsigned int frameSize = reshaped.total() * reshaped.elemSize(); // Frame size is 307200 (640x480)
    17. qint64 bytes;
    18.  
    19. while(1)
    20. {
    21. if(aborted) //Aborted is a global variable
    22. {
    23. socket.disconnectFromHost();
    24. break;
    25. }
    26.  
    27. QMutexLocker locker(&mutex);
    28. if(isDataReady == 1)
    29. {
    30. qDebug() << "Writing frame to socket...";
    31. bytes = socket.write((char*)reshaped.data, frameSize);
    32. socket.waitForBytesWritten();
    33. isDataReady = 0;
    34. }
    35. locker.unlock();
    36. QThread::usleep(100);
    37. }
    38. }
    To copy to clipboard, switch view to plain text mode 

    The frames are written correctly to the socket, I can visualize them with a server written in pure C++ and pthreads but I can not do the same using Qt ( the reason of this post )

    At the server side (a really basic implementation) I accept a client connect and I try to get the bytes sent from the client

    Qt Code:
    1. // FrameServer.cpp
    2. FramesServer::FramesServer(QObject *parent)
    3. : QObject(parent)
    4. {
    5. m_server = new QTcpServer(this);
    6. connect(m_server, SIGNAL(newConnection()),
    7. this, SLOT(newConnection()));
    8.  
    9. if(!m_server->listen(QHostAddress::Any, 2222))
    10. {
    11. qDebug() << "Server could not start";
    12. }
    13. else
    14. {
    15. qDebug() << "Server listening on port 2222";
    16. }
    17.  
    18. m_frame = cv::Mat::zeros(480, 640, CV_8UC1);
    19. m_frameSize = m_frame.total() * m_frame.elemSize(); // Frame size is always 307200
    20.  
    21. m_bytesReceived = 0;
    22. m_frameBuffer.clear();
    23. }
    24.  
    25. void FramesServer::newConnection()
    26. {
    27. m_socket = m_server->nextPendingConnection();
    28.  
    29. connect(m_socket, SIGNAL(disconnected()),
    30. m_socket, SLOT(deleteLater()));
    31.  
    32. connect(m_socket, SIGNAL(readyRead()),
    33. this, SLOT(readFrame()));
    34. }
    35.  
    36. void FramesServer::readFrame()
    37. {
    38. // Get the data
    39.  
    40. while((m_bytesReceived = m_socket->bytesAvailable()) > 0)
    41. {
    42. m_frameBuffer.append(m_socket->readAll());
    43. if(m_bytesReceived == m_frameSize) // I received a complete frame so I want to display it
    44. {
    45. qDebug() << m_frameBuffer.size();
    46. for (int i = 0; i < img.rows; ++i)
    47. {
    48. for (int j = 0; j < img.cols; ++j)
    49. {
    50. (img.row(i)).col(j) = (uchar)m_frameBuffer.at(((img.cols) * i) + j);
    51. cv::imshow("DISPLAY", img);
    52. }
    53. }
    54. m_frameBuffer.clear();
    55. }
    56. }
    57. }
    To copy to clipboard, switch view to plain text mode 
    From the Qt doc I understood that the readyRead signal is fired every time that there is available data for reading so I connected to the slot readFrame thar read data and try to complete a frame ( 307200 bytes )
    Unfortunately the m_frameBuffer not always shows 307200 value, most of time is a different value and I can not construct correctly the cv::Mat image so I can not display it.

    I don't know if the logic is well implemented, I would receive a help on it

    Thank you
    Franco Amato

  2. #2
    Join Date
    Mar 2008
    Location
    Kraków, Poland
    Posts
    1,410
    Thanked 256 Times in 251 Posts
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default Re: OpenCV image sending through Qt socket

    FrameServer::readFrame should look something like this
    Qt Code:
    1. void FramesServer::readFrame()
    2. {
    3. // Get the data
    4.  
    5. while(m_socket->bytesAvailable() > 0)
    6. {
    7. m_frameBuffer.append(m_socket->readAll());
    8. if(m_frameBuffer.size() >= m_frameSize) // I received a complete frame so I want to display it
    9. {
    10. QByteArray workBuffer = m_frameBuffer.left(m_frameSize);
    11. for (int i = 0; i < img.rows; ++i)
    12. {
    13. for (int j = 0; j < img.cols; ++j)
    14. {
    15. (img.row(i)).col(j) = (uchar)workBuffer.at(((img.cols) * i) + j);
    16. cv::imshow("DISPLAY", img);
    17. }
    18. }
    19. m_frameBuffer.remove(0,m_frameSize);
    20. }
    21. }
    22. }
    To copy to clipboard, switch view to plain text mode 
    Think about why.

  3. #3
    Join Date
    Nov 2007
    Location
    Italy
    Posts
    615
    Thanks
    22
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default Re: OpenCV image sending through Qt socket

    Hi Lesiok,
    it worked. I only had to change the code to convert from QByteArray to cv::Mat in this way:

    Qt Code:
    1. QByteArray workBuffer = m_frameBuffer.left(m_frameSize);
    2. cv::Mat frame = cv::Mat(480, 640, CV_8UC1, workBuffer.data());
    3. if(!frame.empty())
    4. {
    5. // Display it on an OpenCV window
    6. imshow("DISPLAY", frame);
    7. }
    To copy to clipboard, switch view to plain text mode 

    For some reason that I don't understand yet the previous conversion code gave as result an empty image but still having the right size.

    Thank you
    Franco Amato

Similar Threads

  1. Sending structure throurh TCP socket
    By veeresh in forum Qt Programming
    Replies: 1
    Last Post: 4th June 2013, 14:12
  2. sending quint32 variable in socket
    By sksingh73 in forum Newbie
    Replies: 1
    Last Post: 24th June 2010, 01:56
  3. QUdpSocket: sending and receiving on the same socket?
    By mhoover in forum Qt Programming
    Replies: 16
    Last Post: 17th June 2009, 03:28
  4. sending structure through udp socket programming
    By hemrajn in forum Qt Programming
    Replies: 9
    Last Post: 15th May 2009, 23:55
  5. sending int array through socket
    By babu198649 in forum Qt Programming
    Replies: 5
    Last Post: 20th December 2007, 08:16

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.