QTcpSocket readyRead strange behavior

[naroin]

thats what i do.

i ready 2 bytes, that should contain the packet size (here always 19998), then i read the buffer.
As a consequence, i should always read 19998 as the size, but i dont.

[wysota]

i ready 2 bytes, that should contain the packet size (here always 19998), then i read the buffer.

"packet" is the term reserved for Internel Protocol (IP). TCP works in a concept of "segments". So 19998 is definitely not the packet size. You are sending a stream of 19998 bytes, that's it. You never know across how many packets or segments the data spans.

As a consequence, i should always read 19998 as the size, but i dont.

No, you shouldn't. Eventually you will get 20000 (2+19998) bytes at the receiving side but there is no guarantee they will arrive at the same time.

[naroin]

ok, sorry, my terms are not very clear
19998 is my DATA size.
i'm sending a stream of 20000 bytes, the first 2 containing the size of the rest of the data (that is, 19998).

i know my data won't arrive at the same time, and if you look at the code, i verify that i have enough data before reading it.
If i do it wrong, i really don't know how i can do right.

What should i change in my code to make it works?

[high_flyer]

What should i change in my code to make it works?

Do what I answered you in my previous post!

[wysota]

Well, I would definitely not cast char* to short like that. It is possible the sending side doesn't send it the way you expect it and you get bogus data.

Try something like:

QByteArray buffer; // somewhere
void X::onReadyRead() {
  buffer += socket->readAll(); // read all pending data
 
  while(buffer.size()>=2){
    quint16 size = (buffer.at(0) << 8) + buffer.at(1); // 16bit unsigned value
    if(buffer.size() <= 2+size) return;
    QByteArray data = buffer.mid(2, size);
    processData(data);
    buffer.remove(0,2+size);
  }
}

QByteArray buffer; // somewhere
void X::onReadyRead() {
  buffer += socket->readAll(); // read all pending data

  while(buffer.size()>=2){
    quint16 size = (buffer.at(0) << 8) + buffer.at(1); // 16bit unsigned value
    if(buffer.size() <= 2+size) return;
    QByteArray data = buffer.mid(2, size);
    processData(data);
    buffer.remove(0,2+size);
  }
}

To copy to clipboard, switch view to plain text mode 

Of course adjust the sending side to send the size in the same manner.

[naroin]

high_flyer i really dont understand, i'm sorry if i am stupid, i DONT ASSUME that all data arrives if SOME data arrives.
Check the code:

qint64 nBytes = pSocket->bytesAvailable();
 
	if (nBytes < 2)
		return;
	pSocket->peek((char*)&sSize, 2);
 
if (nBytes < 2 + sSize)
	{
		qDebug()<<"  not enough data in stream";
		return;
	}

qint64 nBytes = pSocket->bytesAvailable();
 
	if (nBytes < 2)
		return;
	pSocket->peek((char*)&sSize, 2);

if (nBytes < 2 + sSize)
	{
		qDebug()<<"  not enough data in stream";
		return;
	}

To copy to clipboard, switch view to plain text mode 

---

Added after 32 minutes:

wysota, i've got exactly the same problem.

Sometimes, i read a size of 0 instead of 19998.
i've changed the content of my data
memset(data, 1, 19998);
then the size i read is 257. (0x0101)

as a consequence, there seems to be a shift in stream. Can it be due to TCP retransmission on error?
Windows socket bug? Qt bug?

thanks for your help.

[naroin]

no more help?

[high_flyer]

I don't understand what help you want.
Can you show your current code?
Do you read until you get the buffer in the size you expect?

[naroin]

yes i read until i get the buffer in the size i expect

the code is in my first post... where is it false??

void Client:nSocketReadyRead()

[high_flyer]

Your code in the first post does NOT wait for the expected size:

qint64 nBytes = pSocket->bytesAvailable(); //you check bytes available
 
	qDebug()<<"onSocketReadyRead bytes="<<nBytes;
 
	if (nBytes < 2) // it could be you get 0 or 1 bytes, so you get out here. But lets say you got 3, so we go on.
		return;
	qDebug()<<"Bytes="<<nBytes;
	pSocket->peek((char*)&sSize, 2);//why do you use peek and not bytesAvailable? also, why do you peel only 2 bytes ahead? it will alway be at most 2 bytes this the following if will always get out.
 
	qDebug()<<"Packet size="<<sSize;
 
if (sSize != 19998) //so we didn't read 19998, (sSize will be at most 2 because of the peek) which is the size you are waiting for, so we get in to the if.
	{
		QMessageBox::critical(NULL, tr("error"), tr("This is it"));
		pSocket->disconnectFromHost();
		return; //And what do you do? you get out!!
	}
 
	if (nBytes < 2 + sSize)
	{
		qDebug()<<"  not enough data in stream";
		return;
	}

qint64 nBytes = pSocket->bytesAvailable(); //you check bytes available

	qDebug()<<"onSocketReadyRead bytes="<<nBytes;
	
	if (nBytes < 2) // it could be you get 0 or 1 bytes, so you get out here. But lets say you got 3, so we go on.
		return;
	qDebug()<<"Bytes="<<nBytes;
	pSocket->peek((char*)&sSize, 2);//why do you use peek and not bytesAvailable? also, why do you peel only 2 bytes ahead? it will alway be at most 2 bytes this the following if will always get out.

	qDebug()<<"Packet size="<<sSize;

if (sSize != 19998) //so we didn't read 19998, (sSize will be at most 2 because of the peek) which is the size you are waiting for, so we get in to the if.
	{
		QMessageBox::critical(NULL, tr("error"), tr("This is it"));
		pSocket->disconnectFromHost();
		return; //And what do you do? you get out!!
	}

	if (nBytes < 2 + sSize)
	{
		qDebug()<<"  not enough data in stream";
		return;
	}

To copy to clipboard, switch view to plain text mode 

EDIT:

if (sSize != 19998)

if (sSize != 19998)

To copy to clipboard, switch view to plain text mode 

is a tottaly a problem.
Read the peek documentation and be sure to understand it.
sSize should be a buffer, as long as the the data you want to read.
It does not store the size read size, but the read data!
If at all, you probably then use it like this:

int iSize = peel(buff,maySize);
 
if(iSize != iExpectedSize){...}

int iSize = peel(buff,maySize);

if(iSize != iExpectedSize){...}

To copy to clipboard, switch view to plain text mode 

[naroin]

the buffer i want to stream contains:
2 bytes, containing the size of the buffer
rest of the buffer.
in the final program, the size may differ from one packet to another, so i have to put it at the beginning of the send data.
For my tests here, i put it at 19998.

the goal is to read
* the size (in the 2 first bytes)
* then the buffer, corresponding to size just read
then redo the same.

here's my code with comments:

void Client::onSocketReadyRead()
{
	short sSize;
	char* data;
	unsigned long lPacketIndex;
	qint64 nBytes = pSocket->bytesAvailable();
 
	qDebug()<<"onSocketReadyRead bytes="<<nBytes;
 
	if (nBytes < 2) //we haven't read enough data to get the buffer size, return
		return;
	qDebug()<<"Bytes="<<nBytes;
	pSocket->peek((char*)&sSize, 2); //we read 2 bytes, we get the size of the buffer INSIDE these 2 bytes
//here i use peek to get the 2 bytes containing the size, without removing it from the buffer, because i may have not enough data, and i'll have to redo it.
 
	qDebug()<<"Packet size="<<sSize;
/* this IF is only to get easily the error . It's DEBUG code.
    as i KNOW the size must be 19998, if it's not, there was an error.
*/
	if (sSize != 19998) 
	{
//i put a breakpoint here, the code stops.
		QMessageBox::critical(NULL, tr("error"), tr("This is it"));
		pSocket->disconnectFromHost();
		return;
	}
 
	if (nBytes < 2 + sSize) //here we wait to have enough data to read sSize bytes
	{
		qDebug()<<"  not enough data in stream";
		return;
	}
 
	data = new char[sSize];
 
	pSocket->read((char*)&sSize, 2);
	pSocket->read(data, sSize);
 
	memcpy((char*)&lPacketIndex, data, 4);
 
	qDebug()<<"  data read, packet index="<<lPacketIndex;
 
	delete data;
}

void Client::onSocketReadyRead()
{
	short sSize;
	char* data;
	unsigned long lPacketIndex;
	qint64 nBytes = pSocket->bytesAvailable();

	qDebug()<<"onSocketReadyRead bytes="<<nBytes;
	
	if (nBytes < 2) //we haven't read enough data to get the buffer size, return
		return;
	qDebug()<<"Bytes="<<nBytes;
	pSocket->peek((char*)&sSize, 2); //we read 2 bytes, we get the size of the buffer INSIDE these 2 bytes
//here i use peek to get the 2 bytes containing the size, without removing it from the buffer, because i may have not enough data, and i'll have to redo it.

	qDebug()<<"Packet size="<<sSize;
/* this IF is only to get easily the error . It's DEBUG code.
    as i KNOW the size must be 19998, if it's not, there was an error.
*/
	if (sSize != 19998) 
	{
//i put a breakpoint here, the code stops.
		QMessageBox::critical(NULL, tr("error"), tr("This is it"));
		pSocket->disconnectFromHost();
		return;
	}

	if (nBytes < 2 + sSize) //here we wait to have enough data to read sSize bytes
	{
		qDebug()<<"  not enough data in stream";
		return;
	}

	data = new char[sSize];

	pSocket->read((char*)&sSize, 2);
	pSocket->read(data, sSize);

	memcpy((char*)&lPacketIndex, data, 4);

	qDebug()<<"  data read, packet index="<<lPacketIndex;

	delete data;
}

To copy to clipboard, switch view to plain text mode 

is it clearer?

[high_flyer]

wysota gave you the solution, why don't you use it?

[naroin]

I have answrered : i've exactly the same problem with wysota's solution.

As i already said, it works well for a certain time, then it randomly fail. In addition, it does not fail in a "good" network with no TCP retransmission, and often fails in a "bad" network, like low wifi.

well, i'll use directly windows sockets.

[wysota]

Can we see the code for the sender? Also are the sender and the receiver running the same operating system and the same CPU type?