QTcpSocket write data with packet loss - Idiot proof sample does not work!

[wysota]

So, server side is just fine. The problem is on client side.

That's not so certain. Your TCP stack may be using different TCP options for both connections. Nevertheless I really suggest you use waitForBytesWritten() to make sure it is not at fault. You can always remove it if it doesn't help. If it doesn't work then tap into the stream between the sender and the receiver like I advised a couple of posts ago to see if all data leaves the sender.

Where is that limitation? As I can see there is quint16 (64k) buffer size which gives me 32k files transfer with DataStream serialization.

Ekhm... sorry, it was late when I was writing my post Yes, 64kB - 6B = 65530B. Still this problem (and all others) remains.

What is your suggestion? How to change my protocol at network layer?

It depends on what your application is doing. The most basic thing would be to send the size of the data into the socket (e.g. in network byte order) and then start streaming the data itself chunk by chunk.

[jboban]

tap into the stream between the sender and the receiver

waitForBytesWritten() was already tried without success. How can I use your NetworqDebugger? If I choose SOCKS5 (with modification of my client), I got the content I sent, but in console your program loops with a message "QNativeSocketEngine::write() was not called in QAbstractSocket::ConnectedState" and put CPU load to 100%. Otherwise, NetworqDebugger connect to my server side and send wrong data, eg. wrong packet size and then only 1 data byte.

The most basic thing would be to send the size of the data into the socket (e.g. in network byte order) and then start streaming the data itself chunk by chunk.

I thought that I'm already doing.

[wysota]

If I choose SOCKS5 (with modification of my client), I got the content I sent, but in console your program loops with a message "QNativeSocketEngine::write() was not called in QAbstractSocket::ConnectedState" and put CPU load to 100%.

It might suggest one of the ends disconnects before all the data is written.

Otherwise, NetworqDebugger connect to my server side and send wrong data, eg. wrong packet size and then only 1 data byte.

The proxy doesn't modify your data in any way. If it sends incorrect data, it means it has received incorrect data.

I thought that I'm already doing.

No. You are sending a blob containing two bytes of serialized integer followed by serialized byte array with data. You're not sending any chunks and you're not encoding the data size properly. You're totally ignoring the fact that size may not fit into 16 bits but you're then serializing all of the data that can be of arbitrary size (e.g. 16GB). I'm assuming you're doing the same thing on the other end -- reading two bytes, interpreting it as blob size and then waiting in a loop for this amount of data to come in. At best it will put your peers out of sync, at worst it will DOS your receiver (if no data ever comes in) and possibly your sender (as you may run out of RAM trying to send a very large chunk of data that is much larger than 64kB).

[jboban]

You're totally ignoring the fact that size may not fit into 16 bits but you're then serializing all of the data that can be of arbitrary size (e.g. 16GB).

This is specific case and I have not big files, but you are right. I have to redesign complete protocol and hope then it will work. Thank you very much for help and I will be free to ask for help again.

[wysota]

I hope that when you do that, your remaining problems with the transfer will be solved automatically as a side effect to improving the protocol.