+ Reply to Thread
Results 1 to 3 of 3

Thread: OpenCV image sending through Qt socket

  1. #1
    Join Date
    Nov 2007
    Location
    Italy
    Posts
    615
    Thanks
    22
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default OpenCV image sending through Qt socket

    Hi,
    I'm trying to port to QT, a program that sends webcam frames (cv :: Mat) through Qt socket, like a streaming video. I could do it successfully using C++ and pthreads only but I have issues using the QTcpSocket mechanism.

    The program logic is really simple: at client side i capture webcam frames and I send to the server after connecting.

    In a timer slot I capture the frames, resize to 640x480, convert to gray scale and write to a socket:

    Qt Code:
    1. // The main code of the timer slot
    2. QObject::connect(&captureLoopTimer, &QTimer::timeout, [&]()
    3. {
    4. mutex.lock();
    5. capture.read(frame);
    6. if(frame.empty())
    7. {
    8. qWarning() << "Empty frame";
    9. mutex.unlock();
    10. return;
    11. }
    12. else
    13. {
    14. qDebug() << "Frame received";
    15. }
    16. // I correctly got a webcam frame -> convert to grayscale
    17. flip(frame, frame, 1);
    18. cvtColor(frame, gray, CV_BGR2GRAY);
    19.  
    20. // Frame is ready to be written to a tcp socket
    21. isDataReady = 1;
    22. mutex.unlock();
    23. });
    To copy to clipboard, switch view to plain text mode 

    The frames are written to a QTcpSocket in an external thread launched using a QtConcurrent::run(writeFramesToSocket), following the main code

    Qt Code:
    1. void writeFramesToSocket()
    2. {
    3. QTcpSocket socket;
    4. socket.connectToHost("127.0.0.1", 2222);
    5. if(socket.waitForConnected(5000))
    6. {
    7. qDebug() << "Connected...";
    8. }
    9. else
    10. {
    11. qWarning() << "Can't connect to host";
    12. return;
    13. }
    14.  
    15. reshaped = (gray.reshape(0, 1)); // To make it continuous
    16. unsigned int frameSize = reshaped.total() * reshaped.elemSize(); // Frame size is 307200 (640x480)
    17. qint64 bytes;
    18.  
    19. while(1)
    20. {
    21. if(aborted) //Aborted is a global variable
    22. {
    23. socket.disconnectFromHost();
    24. break;
    25. }
    26.  
    27. QMutexLocker locker(&mutex);
    28. if(isDataReady == 1)
    29. {
    30. qDebug() << "Writing frame to socket...";
    31. bytes = socket.write((char*)reshaped.data, frameSize);
    32. socket.waitForBytesWritten();
    33. isDataReady = 0;
    34. }
    35. locker.unlock();
    36. QThread::usleep(100);
    37. }
    38. }
    To copy to clipboard, switch view to plain text mode 

    The frames are written correctly to the socket, I can visualize them with a server written in pure C++ and pthreads but I can not do the same using Qt ( the reason of this post )

    At the server side (a really basic implementation) I accept a client connect and I try to get the bytes sent from the client

    Qt Code:
    1. // FrameServer.cpp
    2. FramesServer::FramesServer(QObject *parent)
    3. : QObject(parent)
    4. {
    5. m_server = new QTcpServer(this);
    6. connect(m_server, SIGNAL(newConnection()),
    7. this, SLOT(newConnection()));
    8.  
    9. if(!m_server->listen(QHostAddress::Any, 2222))
    10. {
    11. qDebug() << "Server could not start";
    12. }
    13. else
    14. {
    15. qDebug() << "Server listening on port 2222";
    16. }
    17.  
    18. m_frame = cv::Mat::zeros(480, 640, CV_8UC1);
    19. m_frameSize = m_frame.total() * m_frame.elemSize(); // Frame size is always 307200
    20.  
    21. m_bytesReceived = 0;
    22. m_frameBuffer.clear();
    23. }
    24.  
    25. void FramesServer::newConnection()
    26. {
    27. m_socket = m_server->nextPendingConnection();
    28.  
    29. connect(m_socket, SIGNAL(disconnected()),
    30. m_socket, SLOT(deleteLater()));
    31.  
    32. connect(m_socket, SIGNAL(readyRead()),
    33. this, SLOT(readFrame()));
    34. }
    35.  
    36. void FramesServer::readFrame()
    37. {
    38. // Get the data
    39.  
    40. while((m_bytesReceived = m_socket->bytesAvailable()) > 0)
    41. {
    42. m_frameBuffer.append(m_socket->readAll());
    43. if(m_bytesReceived == m_frameSize) // I received a complete frame so I want to display it
    44. {
    45. qDebug() << m_frameBuffer.size();
    46. for (int i = 0; i < img.rows; ++i)
    47. {
    48. for (int j = 0; j < img.cols; ++j)
    49. {
    50. (img.row(i)).col(j) = (uchar)m_frameBuffer.at(((img.cols) * i) + j);
    51. cv::imshow("DISPLAY", img);
    52. }
    53. }
    54. m_frameBuffer.clear();
    55. }
    56. }
    57. }
    To copy to clipboard, switch view to plain text mode 
    From the Qt doc I understood that the readyRead signal is fired every time that there is available data for reading so I connected to the slot readFrame thar read data and try to complete a frame ( 307200 bytes )
    Unfortunately the m_frameBuffer not always shows 307200 value, most of time is a different value and I can not construct correctly the cv::Mat image so I can not display it.

    I don't know if the logic is well implemented, I would receive a help on it

    Thank you
    Franco Amato

  2. #2
    Join Date
    Mar 2008
    Location
    Kraków, Poland
    Posts
    1,372
    Thanked 254 Times in 249 Posts
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default Re: OpenCV image sending through Qt socket

    FrameServer::readFrame should look something like this
    Qt Code:
    1. void FramesServer::readFrame()
    2. {
    3. // Get the data
    4.  
    5. while(m_socket->bytesAvailable() > 0)
    6. {
    7. m_frameBuffer.append(m_socket->readAll());
    8. if(m_frameBuffer.size() >= m_frameSize) // I received a complete frame so I want to display it
    9. {
    10. QByteArray workBuffer = m_frameBuffer.left(m_frameSize);
    11. for (int i = 0; i < img.rows; ++i)
    12. {
    13. for (int j = 0; j < img.cols; ++j)
    14. {
    15. (img.row(i)).col(j) = (uchar)workBuffer.at(((img.cols) * i) + j);
    16. cv::imshow("DISPLAY", img);
    17. }
    18. }
    19. m_frameBuffer.remove(0,m_frameSize);
    20. }
    21. }
    22. }
    To copy to clipboard, switch view to plain text mode 
    Think about why.

  3. #3
    Join Date
    Nov 2007
    Location
    Italy
    Posts
    615
    Thanks
    22
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default Re: OpenCV image sending through Qt socket

    Hi Lesiok,
    it worked. I only had to change the code to convert from QByteArray to cv::Mat in this way:

    Qt Code:
    1. QByteArray workBuffer = m_frameBuffer.left(m_frameSize);
    2. cv::Mat frame = cv::Mat(480, 640, CV_8UC1, workBuffer.data());
    3. if(!frame.empty())
    4. {
    5. // Display it on an OpenCV window
    6. imshow("DISPLAY", frame);
    7. }
    To copy to clipboard, switch view to plain text mode 

    For some reason that I don't understand yet the previous conversion code gave as result an empty image but still having the right size.

    Thank you
    Franco Amato

+ Reply to Thread

Similar Threads

  1. Sending structure throurh TCP socket
    By veeresh in forum Qt Programming
    Replies: 1
    Last Post: 4th June 2013, 14:12
  2. sending quint32 variable in socket
    By sksingh73 in forum Newbie
    Replies: 1
    Last Post: 24th June 2010, 01:56
  3. QUdpSocket: sending and receiving on the same socket?
    By mhoover in forum Qt Programming
    Replies: 16
    Last Post: 17th June 2009, 03:28
  4. sending structure through udp socket programming
    By hemrajn in forum Qt Programming
    Replies: 9
    Last Post: 15th May 2009, 23:55
  5. sending int array through socket
    By babu198649 in forum Qt Programming
    Replies: 5
    Last Post: 20th December 2007, 08:16

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts