cannot read correct len from socket

[wysota]

1. sizeof(char) != 4
2. int does not contain decimal digits, it is a binary value ranging from ~ -2^31 to 2^31 (which can hold 10 decimal digits) for a 32 bit integer memory layout
3. reading a value that has not been written cannot end well, can it?
4. you're doing a lot of other errors in your code which completely prove you have no idea what you are doing and just blindly trying to get things working without any understanding what is going on.

Sockets are not magical entities, they do not convert your data, add some artificial data or remove some data -- if you write a sequence of bytes to it, the exact same sequence of bytes will be read by the other end of the communication channel. Don't try to do network programming if you don't understand how it works. Spend two days reading about network programming instead of listening to hints you don't even understand.

[saman_artorious]

Then why are you trying to read them as an integer?

I made a mistake. it is not as chars. it is binary data. So, I need to read the first 4 bytes to get the size of buffer.
The problem is that I cannot simply write an integer to the socket, and read the int 4 bytes in the client.

[wysota]

The problem is that I cannot simply write an integer to the socket, and read the int 4 bytes in the client.

Hmmm... why not?

[saman_artorious]

Then why are you trying to read them as an integer?

I want to read it as binary. appending data length at the beginning with this:

void tcpClient::sendToServer()
{
    QByteArray data;
 
    data.append(5);
    data.append("Saman");
 
    if(clientSocket->write (data) == -1)
    {
        qDebug("write() failed");
    }
}

void tcpClient::sendToServer()
{
    QByteArray data;

    data.append(5);
    data.append("Saman");

    if(clientSocket->write (data) == -1)
    {
        qDebug("write() failed");
    }
}

To copy to clipboard, switch view to plain text mode 

in the server, let's read the first 4 bytes, as the size of integer and then read the text.

QByteArray         text;
 
    int length;
 
    clientConnection->read((char*)&length, 4);
 
    qDebug() << length;
 
    text.reserve(length);
 
    clientConnection->read(text.data(), length);
 
    clientConnection->flush();

QByteArray         text;

    int length;

    clientConnection->read((char*)&length, 4);

    qDebug() << length;

    text.reserve(length);

    clientConnection->read(text.data(), length);
    
    clientConnection->flush();

To copy to clipboard, switch view to plain text mode 

output:
length read is 1835094789 !

what am I missing?

[Lesiok]

QByteArray::append(5) appends one byte with a value of 5 not integer. Do it in such a way :

clientSocket->write (5);//You are sending int(5) and on other side You can read int
clientSocket->write("Saman");

clientSocket->write (5);//You are sending int(5) and on other side You can read int
clientSocket->write("Saman");

To copy to clipboard, switch view to plain text mode 

[saman_artorious]

wrong! that would prompt invalid conversion from int to const char*

[wysota]

I want to read it as binary.

You are always reading it as binary, even if it is textual data. It is just a matter of interpreting the data.

what am I missing?

You are missing a couple of hours of learning about computers, their memory and data layout of different data types.

1835094789d = 0x6D615305 = 0x0553616D (swapped byte order) = "\5Sam" interpreted as text. Does that ring a bell?

[ChrisW67]

QByteArray::append(5) appends one byte with a value of 5 not integer. Do it in such a way :

clientSocket->write (5);//You are sending int(5) and on other side You can read int

clientSocket->write (5);//You are sending int(5) and on other side You can read int

To copy to clipboard, switch view to plain text mode 

There's no write(int) function anywhere I can see in the socket/qiodevice interfaces; this won't compile.

[Lesiok]

OK, clientSocket->write(5) is only idea. This code should compile

QByteArray data;
QDataStream m_stream(&data,QIODevice::WriteOnly);
m_stream << 5 << "Saman";
clientSocket->write(data);

QByteArray data;
QDataStream m_stream(&data,QIODevice::WriteOnly);
m_stream << 5 << "Saman";
clientSocket->write(data);

To copy to clipboard, switch view to plain text mode 

saman_artorious read more about QDataStream.

[wysota]

saman_artorious read more about QDataStream.

No, please, don't read about QDataStream. Ignore its existance completely for this project.

[saman_artorious]

I did not know about QDataStream.

Now, it reads the integer fine. However, the amount of data it reads does not depend on how much I allocate according to length.
it always reads the whole thing. How can I control it?

void Server::readState()
{
   int len = 0;
 
   QDataStream in(clientConnection);
 
   in >> len;
 
   qDebug() << len;
 
   QByteArray data;
 
   data.reserve(len);
 
   in >> data;         // read 20 chars and not all
 
   qDebug() << data.data();
 
   data.clear();
 
   clientConnection->flush();
}

void Server::readState()
{
   int len = 0;

   QDataStream in(clientConnection);

   in >> len;

   qDebug() << len;

   QByteArray data;

   data.reserve(len);

   in >> data;         // read 20 chars and not all

   qDebug() << data.data();

   data.clear();

   clientConnection->flush();
}

To copy to clipboard, switch view to plain text mode 

what I sent is of length 50, but only 20 chars would be read, though, it reads all 50 each time:

void tcpClient::sendToServer()
{
    QByteArray data;
    QDataStream m_stream(&data,QIODevice::WriteOnly);
    m_stream << 20 << "xxxxxxxxxx yyyyyyyyyy wwwwwwwwww zzzzzzzzzz aaaaaaaDD"; //text size = 50
    clientSocket->write(data);
 
    clientSocket->flush();
}

void tcpClient::sendToServer()
{
    QByteArray data;
    QDataStream m_stream(&data,QIODevice::WriteOnly);
    m_stream << 20 << "xxxxxxxxxx yyyyyyyyyy wwwwwwwwww zzzzzzzzzz aaaaaaaDD"; //text size = 50
    clientSocket->write(data);

    clientSocket->flush();
}

To copy to clipboard, switch view to plain text mode 

output:
20
xxxxxxxxxx yyyyyyyyyy wwwwwwwwww zzzzzzzzzz aaaaaaaDD

[yeye_olive]

I did not know about QDataStream.

Now, it reads the integer fine. However, the amount of data it reads does not depend on how much I allocate according to length.
it always reads the whole thing. How can I control it?

void Server::readState()
{
   int len = 0;
 
   QDataStream in(clientConnection);
 
   in >> len;
 
   qDebug() << len;
 
   QByteArray data;
 
   data.reserve(len);
 
   in >> data;         // read 20 chars and not all
 
   qDebug() << data.data();
 
   data.clear();
 
   clientConnection->flush();
}

void Server::readState()
{
   int len = 0;

   QDataStream in(clientConnection);

   in >> len;

   qDebug() << len;

   QByteArray data;

   data.reserve(len);

   in >> data;         // read 20 chars and not all

   qDebug() << data.data();

   data.clear();

   clientConnection->flush();
}

To copy to clipboard, switch view to plain text mode 

what I sent is of length 50, but only 20 chars would be read, though, it reads all 50 each time:

void tcpClient::sendToServer()
{
    QByteArray data;
    QDataStream m_stream(&data,QIODevice::WriteOnly);
    m_stream << 20 << "xxxxxxxxxx yyyyyyyyyy wwwwwwwwww zzzzzzzzzz aaaaaaaDD"; //text size = 50
    clientSocket->write(data);
 
    clientSocket->flush();
}

void tcpClient::sendToServer()
{
    QByteArray data;
    QDataStream m_stream(&data,QIODevice::WriteOnly);
    m_stream << 20 << "xxxxxxxxxx yyyyyyyyyy wwwwwwwwww zzzzzzzzzz aaaaaaaDD"; //text size = 50
    clientSocket->write(data);

    clientSocket->flush();
}

To copy to clipboard, switch view to plain text mode 

output:
20
xxxxxxxxxx yyyyyyyyyy wwwwwwwwww zzzzzzzzzz aaaaaaaDD

Please stop trying things at random, this is getting out of hand. You went ahead with QDataStream; you should have read its documentation. Do you have any idea what the insertions (<<) and extractions (>>) do?

What you really need is read a bit about serialization and deserialization of basic C/C++ types. Forget about the network for now, just pretend you are designing a platform and architecture-independent binary file format in which to save/from which to load the piece of data you are interested in. This has nothing to do with Qt, by the way.

[ChrisW67]

Now, it reads the integer fine. However, the amount of data it reads does not depend on how much I allocate according to length.
it always reads the whole thing. How can I control it?

Yes, it is doing that because that is the correct behaviour for QDataStream. You don't "control it" but you could at least try to understand it and all that has preceded it in this thread.

If you are serious about understanding then try this experiment: using each approach write the data to a file (substitute a QFile for the QTcpSocket). These files represent exactly what you are sending over the wire. Look at the length of the files. Inspect the files byte by byte (a binary editor or Linux od command). Try changing the numbers and see which bytes change. Look at sizeof(int) how many bytes are you trying to read?

[saman_artorious]

Yes, it is doing that because that is the correct behaviour for QDataStream. You don't "control it" but you could at least try to understand it and all that has preceded it in this thread.

If you are serious about understanding then try this experiment: using each approach write the data to a file (substitute a QFile for the QTcpSocket). These files represent exactly what you are sending over the wire. Look at the length of the files. Inspect the files byte by byte (a binary editor or Linux od command). Try changing the numbers and see which bytes change. Look at sizeof(int) how many bytes are you trying to read?

It is not like I have new to socket programming. I have done it with C many times. The problem when I even try to send and receive data with simple read n write operations it does not give the expected value. I opened a binary file and read the contents, it writes to buffer correctly. the buffer size is also equal to image size. However, writing it to the other side and reading, is not correct.

QString fileName = "/home/saman/Desktop/1.png";
    QFile file(fileName);
    if (!file.open(QIODevice::ReadOnly)) return;
    QByteArray blob = file.readAll();
 
    int len = blob.length();
    clientSocket->write(blob, len);

QString fileName = "/home/saman/Desktop/1.png";
    QFile file(fileName);
    if (!file.open(QIODevice::ReadOnly)) return;
    QByteArray blob = file.readAll();

    int len = blob.length();
    clientSocket->write(blob, len);

To copy to clipboard, switch view to plain text mode 

server:

int len;
clientConnection->read((char*)&len, 4);
qDebug() << len;

int len;
clientConnection->read((char*)&len, 4);
qDebug() << len;

To copy to clipboard, switch view to plain text mode 

does not read correctly!

[wysota]

It is not like I have new to socket programming. I have done it with C many times.

Then do it with regular C calls, nobody forces you to use Qt classes for that. With plain C calls you'll be having the exact same "problems" you have now -- you will have to format the data somehow and read it somehow according to all rules that govern asynchronous communication between two autonomous systems.

[saman_artorious]

how to format it? what are those rules?

---

Added after 1 11 minutes:

Then do it with regular C calls, nobody forces you to use Qt classes for that. With plain C calls you'll be having the exact same "problems" you have now -- you will have to format the data somehow and read it somehow according to all rules that govern asynchronous communication between two autonomous systems.

I think you are wrong. As I tested the program in another system, with standard c++ rcv function, it could receive the image.
So, that means the send or write commands is correct and client program works fine:

clientSocket->write((char*)&size, 4);
    clientSocket->write(blob.data(), size);

clientSocket->write((char*)&size, 4);
    clientSocket->write(blob.data(), size);

To copy to clipboard, switch view to plain text mode 

However, this shows the server program written (the first one) is not correct, which is quite strange. As I debugged it, it received the packet up to 50 KB and then returns 0! On the other, the other one which used std sockets, can read all of it.

int size;
 clientConnection->read((char*)&size, 4);
 qDebug() << size;
 
 char* image;
 
 qDebug() << size;
 
 image = new char[size];
 
 int size2 = size;
 int offset = 0;
 
 while(size2 > 0)
 {
     int r = clientConnection->read(image+offset, size2);
     if(r == 0)
         break;
     size2 -= r;
     offset += r;
 }

int size;
 clientConnection->read((char*)&size, 4);
 qDebug() << size;

 char* image;

 qDebug() << size;

 image = new char[size];

 int size2 = size;
 int offset = 0;

 while(size2 > 0)
 {
     int r = clientConnection->read(image+offset, size2);
     if(r == 0)
         break;
     size2 -= r;
     offset += r;
 }

To copy to clipboard, switch view to plain text mode 

[wysota]

how to format it? what are those rules?

You should know, it's not the first time you do socket communication.

As I tested the program in another system, with standard c++ rcv function, it could receive the image.

Good for you. Seems the problem is solved, isn't it?

[saman_artorious]

I also tried the receiving buffer code in C. it again only read 49KB and then returns 0, ignoring the rest of file.
I never had such problem in programs, simple send and recv would do it all.
Could I again ask you what I need to do to solve this problem?

[yeye_olive]

I am at a loss. Are you assuming that if you write n bytes using one call to write(), then the other side will read those n bytes in one call to read()? Since you have already programmed with sockets, you must know that TCP is a stream protocol and does not work this way. I recommend checking out http://blog.stephencleary.com/2009/0...ckets-faq.html for more information on this topic.

I suppose I would do something along these lines to solve your problem:

On the sender's side:
1. Send the length of the byte array as e.g. a 32-bit unsigned integer in big-endian byte order (see quint32 and qToBigEndian()).
2. Send the bytes themselves.

On the receiver's side:
1. Store received bytes in a buffer until I have 4 bytes.
2. Interpret those 4 bytes as a 32-bit unsigned integer in big-endian byte order (let n be the value thus read).
3. Store received bytes in a byte array until I have n of them.

This is a quite standard approach to TCP asynchronous communication. You will probably need a few variables to track the current state on the receiver side (e.g. whether you are currently waiting for bytes that are part of the integer or the byte array, how many bytes you are still waiting for, etc.).

[wysota]

I also tried the receiving buffer code in C. it again only read 49KB and then returns 0, ignoring the rest of file.

How unsurprising. I guess if you streamed 16GB of data into the socket, you'd expect to receive that 16GB of data after 2 seconds on the receiving end, right?

I never had such problem in programs, simple send and recv would do it all.

Which means you have no understanding of network programming.

Could I again ask you what I need to do to solve this problem?

You need a good book on networking theory.