Wireshark tell me. Here is the screenshot. If you need I can send you the log file for Wireshark, too.
tcpDump1.jpg
If a segment was missed, it would be retransmitted. I think your log only tells you that wireshark has missed it, not that it wasn't transferred. If you have full conversation between the server and the client logged, you can verify that by returning ack segments.
jboban (26th March 2013)
Yes, but my server side application has missed it, too. That's my problem. Every facts tell me that the data was not arrive to the server. And, when I add just one extra char into source XML file, then he was sent. Really strange. Even that, when binary dump of TCP buffer send with dd it was send successful.
Which byte is missing? The first byte of the size, the first byte of the string payload, the last byte of either, some other byte? Do you insert and receive a correct size figure? Is there an off-by-one error in the receiver?
No byte, but packet of bytes is missed. Packet size was correct received. On client side I have:
Qt Code:
sendMsgImpl: Sent 9198, Rest: 0To copy to clipboard, switch view to plain text mode
And on server side:
Qt Code:
25.03.2013 22:56:30 onReadData::blockSize: 9196, Socket: 21 25.03.2013 22:56:30 onReadData::bytesAvailable: 1406 25.03.2013 22:56:30 onReadData::bytesAvailable: 2814 25.03.2013 22:56:30 onReadData::bytesAvailable: 4222 25.03.2013 22:56:30 onReadData::bytesAvailable: 5630 25.03.2013 22:56:30 onReadData::bytesAvailable: 7038To copy to clipboard, switch view to plain text mode
Then waits and waits and timeouts... Never reached the size.
Now, when I add just one extra char into source XML file on client side, the file sent fine and on server side I'v got:
Qt Code:
25.03.2013 23:08:41 onReadData::blockSize: 9198, Socket: 20 25.03.2013 23:08:41 onReadData::bytesAvailable: 1406 25.03.2013 23:08:41 onReadData::bytesAvailable: 2814 25.03.2013 23:08:41 onReadData::bytesAvailable: 4222 25.03.2013 23:08:41 onReadData::bytesAvailable: 5630 25.03.2013 23:08:41 onReadData::bytesAvailable: 7038 25.03.2013 23:08:41 onReadData::bytesAvailable: 8446 25.03.2013 23:08:41 onReadData::bytesAvailable: 9198To copy to clipboard, switch view to plain text mode
Last edited by jboban; 25th March 2013 at 22:11.
Since you are getting data on the receive end, no packet is missed.
Where does the value from "blockSize" come from? Is it the first two bytes you read from the socket? Do you actually read any data from the socket?
By the way, your client code is potentially wrong in this regard that you disconnect from the peer without waiting for the data to be written to the device. The fact that write() returns doesn't mean the data was actually sent from your app. Either use the bytesWritten() signal or waitForBytesWritten() blocking call.
jboban (26th March 2013)
That's the case in second case, when I manually add one extra char and resize source file with +1.
Yes, first two bytes are blockSize:
I actualy wait for blockSize data to be available and then read the whole message. Look at my last post onReadData::bytesAvailable. It's on server side in slot onReadData() connected to readyRead() socket signal.Qt Code:
out << (quint16)0; out << m_sMsg; out.device()->seek(0); out << (quint16)(bArrMsg.size() - (int)sizeof(quint16)); // <-- Here is the blockSizeTo copy to clipboard, switch view to plain text mode
I'v tried that too, but got the same result. You wrong I guess, in non-blocking TCP communication I don't need to wait for bytes to be written. The manual for disconnectFromHost() says:
Attempts to close the socket. If there is pending data waiting to be written, QAbstractSocket will enter ClosingState and wait until all data has been written.
Last edited by jboban; 25th March 2013 at 23:35.
No. That's the case when application on the receiving end of a TCP stream receives data later in the stream than the supposedly "missed" data. We're talking transport layer here. What happens in the application layer is a different issue and wireshark has nothing to do with this since it doesn't understand your application protocol. I don't know how detailed your knowledge of the networking stack is but it is important what terms you use to refer to protocol data units -- "packets" are data units of the network layer (i.e. Internet Protocol (IP)), "segments" are data units of the TCP protocol (transport layer), let's not mix the two. Thus I can assure you no packets nor segments are missing from your TCP stream. It could be that the last segments were not received by the peer but that's easy to verify using wireshark - the sender should get a TCP segment with ACK flag set to the value of its own initial SYN incremented by the size of data sent, possibly followed by exchange of packets with a FIN flag set that closes the connection. You can use wireshark to check the last ACK segment you get from the server to see how much data was accepted by the server.
That's the sending side, I'm asking about the receiving side.Yes, first two bytes are blockSize:
Qt Code:
out << (quint16)0; out << m_sMsg; out.device()->seek(0); out << (quint16)(bArrMsg.size() - (int)sizeof(quint16)); // <-- Here is the blockSizeTo copy to clipboard, switch view to plain text mode
It doesn't mean you are reading any data. We only know it sits in your socket. If the receiving buffer is full, transmission is halted until space becomes available. Thus it is important that you actually read data from the socket. Since we don't have access to the code of your reader, I'm asking you whether you are calling read (or readAll) on the socket anywhere after you receive the readyRead() signal.I actualy wait for blockSize data to be available and then read the whole message. Look at my last post onReadData::bytesAvailable: ... It's on server side in slot onReadData() connected to socket readyRead() signal.
A totally separate thing is that you're mimicing inherently broken code of the fortune cookie server example that is not fit to be treated as a generic data transfer protocol. The code will simply fail to work e.g. in a case where the data you're trying to send is larger than 16382 (or even 16378) bytes. Since you're not checking the data size anywhere nor making anything to prevent such situation, your code is inherently broken as well.
Last edited by wysota; 25th March 2013 at 23:57.
jboban (26th March 2013)
Ok, I mean segments, or my data chunks. Now I confirm that "missed" data is the last segment sent. You right about that. Thank you.
Now I changed the server code and actually read all available bytes but all the same happens.
Again, when I try dd if=./tcp.bin > /dev/tcp/<server_ip>/<server_port> everything is fine. So, server side is just fine. The problem is on client side.
tcpDump2.jpg
Where is that limitation? As I can see there is quint16 (64k) buffer size which gives me 32k files transfer with DataStream serialization.
What is your suggestion? How to change my protocol at network layer?
Last edited by jboban; 26th March 2013 at 04:19.
That's not so certain. Your TCP stack may be using different TCP options for both connections. Nevertheless I really suggest you use waitForBytesWritten() to make sure it is not at fault. You can always remove it if it doesn't help. If it doesn't work then tap into the stream between the sender and the receiver like I advised a couple of posts ago to see if all data leaves the sender.
Ekhm... sorry, it was late when I was writing my post Yes, 64kB - 6B = 65530B. Still this problem (and all others) remains.Where is that limitation? As I can see there is quint16 (64k) buffer size which gives me 32k files transfer with DataStream serialization.
It depends on what your application is doing. The most basic thing would be to send the size of the data into the socket (e.g. in network byte order) and then start streaming the data itself chunk by chunk.What is your suggestion? How to change my protocol at network layer?
waitForBytesWritten() was already tried without success. How can I use your NetworqDebugger? If I choose SOCKS5 (with modification of my client), I got the content I sent, but in console your program loops with a message "QNativeSocketEngine::write() was not called in QAbstractSocket::ConnectedState" and put CPU load to 100%. Otherwise, NetworqDebugger connect to my server side and send wrong data, eg. wrong packet size and then only 1 data byte.
I thought that I'm already doing.
I don't believe that. If that was the case, no data would arrive at the server application at all. Not a single byte. And after reaching the number of bytes equal to the window size of your TCP connection (which is a bit less than 1kB) the sending end would halt transmission waiting for acknowledgements from the receiving side and would retransmit all data from the beginning if no such ack was received. The way TCP works, you either get all data from the beginning of the stream up to a certain point in time or you don't get anything. It is not possible that some data in the middle (or in the beginning) gets "skipped".
Have a look at this thread: http://www.qtcentre.org/threads/34082-NetworqDebuggerThat's my problem. Every facts tell me that the data was not arrive to the server. And, when I add just one extra char into source XML file, then he was sent. Really strange. Even that, when binary dump of TCP buffer send with dd it was send successful.
Download the program, build it and run your connection through it (e.g. using the SOCKS5 mode). See what gets sent from your application into the network (the "Local" field). It's best if you set the proxy on a different machine than the one your client operates from, this will make sure the data actually leaves your client machine.
jboban (26th March 2013)
Bookmarks