QTcpSocket::readyRead() lost messages, performance issue?

[Juba]

Hi all,
I have a trouble with QTcpSocket readyRead() signal.
My Qt application act as client and the server send messages in an asyncronous way.
The problem is that I notice that sometimes a message isn't signaled by the readyRead().
The message is always the same message in the sequence, but the problem don't always occours.
I'm sure that message comes because I see the network traffic with wireshark.

Here is my client code:

{
// in class ctor...
  m_socket = new QTcpSocket(this);
  connect(m_socket, SIGNAL(connected()),                           this,SLOT(connected()));
  connect(m_socket, SIGNAL(error(QAbstractSocket::SocketError)),      this,SLOT(error(QAbstractSocket::SocketError)));
  connect(m_socket, SIGNAL(disconnected()),                        this,SLOT(disconnected()));
  connect(m_socket, SIGNAL(readyRead()),                           this,SLOT(receiveMessage()));
}
 
void XmlSocket::receiveMessage()
{
    int bytesRead;
    // READ_BUF_SIZE is defined as 512 and the message I receive are 100 chars...
    char buf[READ_BUF_SIZE];
 
    // read data from socket
    // NB: loop only if we read more than 512 bytes...
    QString msgString;
    do {
        bzero(buf,READ_BUF_SIZE);
 
        bytesRead = m_socket->read(buf, READ_BUF_SIZE);
        if(bytesRead < 0)
            qCritical("Error reading data from socket %d",bytesRead);
        else if(bytesRead == 0)
            qWarning("No more data on socket...");
 
        msgString += QString(buf);
        qDebug("++ %s", qPrintable(msgString));
    } while(bytesRead == READ_BUF_SIZE);
 
    // emit signal with available data
    QDomDocument msg;
    if(!msg.setContent(msgString))
        qWarning("Error setting content for string %s", qPrintable(msgString));
 
    emit messageReceived(msg);
}

{
// in class ctor...
  m_socket = new QTcpSocket(this);
  connect(m_socket, SIGNAL(connected()),                           this,SLOT(connected()));
  connect(m_socket, SIGNAL(error(QAbstractSocket::SocketError)),      this,SLOT(error(QAbstractSocket::SocketError)));
  connect(m_socket, SIGNAL(disconnected()),                        this,SLOT(disconnected()));
  connect(m_socket, SIGNAL(readyRead()),                           this,SLOT(receiveMessage()));
}

void XmlSocket::receiveMessage()
{
    int bytesRead;
    // READ_BUF_SIZE is defined as 512 and the message I receive are 100 chars...
    char buf[READ_BUF_SIZE];

    // read data from socket
    // NB: loop only if we read more than 512 bytes...
    QString msgString;
    do {
        bzero(buf,READ_BUF_SIZE);

        bytesRead = m_socket->read(buf, READ_BUF_SIZE);
        if(bytesRead < 0)
            qCritical("Error reading data from socket %d",bytesRead);
        else if(bytesRead == 0)
            qWarning("No more data on socket...");

        msgString += QString(buf);
        qDebug("++ %s", qPrintable(msgString));
    } while(bytesRead == READ_BUF_SIZE);

    // emit signal with available data
    QDomDocument msg;
    if(!msg.setContent(msgString))
        qWarning("Error setting content for string %s", qPrintable(msgString));

    emit messageReceived(msg);
}

To copy to clipboard, switch view to plain text mode 

I have see that the message that I don't receive is sent just 3 ms after the previous one...
Send time
[28/04 16:33:30:423] message 1, received
[28/04 16:33:30:525] message 2, received
[28/04 16:33:30:528] message 3, NOT RECEIVED
[28/04 16:33:31:290] message 4, received

This could be a performance issue???

Now I try to solve using standard socket, but I wish to use QTcpSocket because they offer a simple API and for portability reason.

I am using Qt 4.7.1 on linux platform.

[mattc]

There are many things wrong with your code, the problem is not related to network traffic.

Please take a look at the Fortune Client example for a better way to read fixed-size blocks of data from a socket:

http://doc.qt.nokia.com/latest/netwo...uneclient.html

The equivalent of your receiveMessage() is Client::readFortune() in client.cpp

[Juba]

Hi mattc and thank you for your reply.

I think that your suggested solution don't solve my problem, because when a message is lost the receiveMessage() isn't called at all... To prove this I have inserted a qDebug line at the firt line in the method and I see that when I lost a message the receiveMessage isn't invoked... (Note that my server isn't written in Qt and I don't know how to set the setVersion for the QDataStream).

All my test are executed in a LAN and the messages length are under 100 characters, so I exclude that is a network issue.

I have re implemented the class using standard socket and after an initial test the new implementation work well. I used standard socket in combination with QThread, but I wish to use a simpler implementation with only QTcpSocket.

No one have noticed the same problem?

[mattc]

Here are two major problems I can see with the code you posted:

1) if readyRead() is called and there are less than 100 bytes in the socket, you
emit an incomplete message. The Fortune Client calls bytesAvailable()
to prevent this.

2) if readyRead() is called with more than 100 bytes, you are "packing" them
together in a single message (and, as above, the last part of the message might be incomplete).
This is weird because you say you are working with messages of 100 bytes each.

Also, when you call QString(buf) you are making two assumption that
I cannot verify:

1) the received data has no embedded zeroes
2) the received data is text encoded with the default encoding (QTextCodec::codecForCStrings())

[Juba]

Ok mattc, thank you for reviewing my code, but your post don't focus the problem...

When readyRead() is emitted I always read the correct messages, the problem is that readRead() seems to be not emitted in certain conditions.
The condition isn't easily reproducible, sometime a the burst of 4 messages is received, sometime the client lose the third message...

[squidge]

Have you tried the different connection options, such as DirectConnection (assuming slot is in same thread as signal), QueuedConnection, etc?

[Juba]

So far I've used the default value, but since signal and slot are in the same thread I will try the Qt::QueuedConnection value!

---

Added after 43 minutes:

I have just tryed using Qt::QueuedConnection, but the performance is worse, now when the problem occours I lost 2 messages...

I remark that using standard socket and QThread for now I haven't see any message lost.

[Juba]

Well, I have notice that the issue is that sometimes the underlying network layer put two xml messages in one read... And the messages are separated by a '\0' char... So thank you mattc for your useful answer.

So even QTcpSocket and the standard socket solution works well... I prefer to use the QTcpSocket class.

Thank you all for your answer.

[wysota]

TCP doesn't have the concept of "messages". The fact that you put two blocks of data 20 bytes each on the sending side, doesn't mean the receiving end will get two blocks 20 bytes each. It might be 1 block of 40 bytes or even 40 blocks of one byte. Thus your do-while loop is flawed.