QTcpServer problem receiving big files

[ruben.rodrigues]

hi all!

I am having some problem send/receiving large data files, 500 some kbytes.

Code:



QDataStream in(tcpSocket);
in.setVersion(QDataStream::Qt_4_5);

forever {
if (nextBlockSize == 0){
if (tcpSocket->bytesAvailable() < sizeof (qint64))
break;
in >> nextBlockSize;
cout << "Block size: " << QString::number(nextBlockSize).toStdString() << endl;
}

if (tcpSocket->bytesAvailable() < nextBlockSize)
{
cout << "Going to break" << endl;
break;
}

...


output:


New connection
DataReceived
Block size: 611164
Going to break
DataReceived
Block size: 34364522594
Going to break
DataReceived
Block size: 28429488043851784
Going to break
DataReceived
Block size: 23925866948264050
Going to break
DataReceived
Block size: 214753804389
Going to break
DataReceived
Block size: 32651449492373607
Going to break
DataReceived
Block size: 29555366483460197
Going to break
DataReceived
Block size: 32088581143265345
Going to break
DataReceived
Block size: 31525678435008617
Going to break
DataReceived
Block size: 27866439313784937
Going to break
DataReceived
Block size: 31244194868822016
Going to break
DataReceived
Block size: 2624568615239740
Going to break
DataReceived
Block size: 13229757704634480
Going to break
DataReceived
Block size: 13229637444436069
Going to break
DataReceived
Block size: 33777284975165545
Going to break
DataReceived
Block size: 28429445101781072
Going to break


the DataReceived output means that the slot readyRead has been triggered, and the first Block size, with "611164 bytes" is the correct number.

I think that I am getting multiple connections but I don't know how to do it better.
I have looked to some examples but all look like mine.

[marcvanriet]

Hi,

How do you know that the first 4 bytes are the length of the file you are receiving, and when the first bytes of the file sent ? Is that on the sending side ?
Anyway, I would do some sanity checking on that length. A size of 33777284975165545 seams unrealistic.

Also, what do you do with the bytes when you are 'going to break' ? If it stays in the buffer, maybe you will read the length from this again the next time.

Regards,
Marc

[ruben.rodrigues]

Hi,

thank for your answer. I got this routine from website's examples and the qt book called "C++ Gui Programming with QT4". I know that the correct value is the 611164 because I check it on the client side.
if I don't do the break the programm never stop, but once again I got this from websites.

But basically there is no limitation to the size right??

[wysota]

Do you read the data from the socket anywhere?

[tbscope]

From my understanding of the posted code, when read in the following order:


QDataStream in(tcpSocket);
in.setVersion(QDataStream::Qt_4_5);
in >> nextBlockSize;
cout << "Block size: " << QString::number(nextBlockSize).toStdString() << endl;

This does not display the size.

[ruben.rodrigues]

Do you read the data from the socket anywhere?

yes, right after.


QDataStream in(tcpSocket);
in.setVersion(QDataStream::Qt_4_5);

forever {
if (nextBlockSize == 0){
if (tcpSocket->bytesAvailable() < sizeof (qint64))
break;
in >> nextBlockSize;
cout << "Block size: " << QString::number(nextBlockSize).toStdString() << endl;
}

if (tcpSocket->bytesAvailable() < nextBlockSize)
{
cout << "Going to break" << endl;
break;
}

in >> _applicationCall;
in >> _classCall;
in >> _methodCall;
in >> _eBusinessObject;
in >> qba;
}

and this works if the qba (QByteArray) is small but when it's large like the one I want to store the server stops. Probably it waits until it receives data = 31525678435008617 bytes which means forever.

[tbscope]

No way.

The server already read the whole file.
You just don't display the size.

[ruben.rodrigues]

the nextBlockSize variable is the size. It is a qint64 calculated on the client size

[wysota]

Well, obviously what you display is not a correct size of the data. I would guess that once you exit the forever loop you exit the scope where "nextBlockSize" is defined and when you enter it back again after a subsequent readyRead() the variable gets reinitialized with random data.

[tbscope]

Those are not real sizes. You either do something wrong saving those sizes, or reading those sizes.

Please, take a calculater and see how many terabytes those sizes are.

[ruben.rodrigues]

Well, obviously what you display is not a correct size of the data. I would guess that once you exit the forever loop you exit the scope where "nextBlockSize" is defined and when you enter it back again after a subsequent readyRead() the variable gets reinitialized with random data.

Exactly. I think that too. I just can't figure out why the I get multiple readyRead signals.

First size is correct. the 600000 and some kbytes because I get the same value in the client side. After that I get all those random numbers that are a lot of terabytes

[wysota]

Exactly. I think that too. I just can't figure out why the I get multiple readyRead signals.
Because in normal circumstances you can't fit 600kB in one IP packet. With 600kB you have to expect even 600k readyRead() signals.

[ruben.rodrigues]

ok. Do you know how to solve this?

I have made some changes and I got some values. The package size is 611000 Bytes and the value I am getting in my first connection is 4336 Bytes. That means I need about 140 connection. How do I overcome this.

[wysota]

ok. Do you know how to solve this?
Start by not copying code you don't understand. The forever loop in your code is already prepared for handling multiple readyRead() signals.


How do I overcome this.
It's not anything you have to overcome.

[ruben.rodrigues]

Start by not copying code you don't understand. The forever loop in your code is already prepared for handling multiple readyRead() signals.


It's not anything you have to overcome.

I meant solve. My bad :D

[wysota]

This is strictly a C++ related problem.

[ruben.rodrigues]

so...Am I realy suppous to receive multiple readyRead signals?

[tbscope]

so...Am I realy suppous to receive multiple readyRead signals?

Yes, you're supposed to get multiple ready read signals.

First, read the entire data. Since it is a datastream, you might want to use the datastream only when it is complete read.

[ruben.rodrigues]

Hi,

I am trying to handle this but is not working and I am running out of ideas. Can someone explaining me how to be sure that all the data is received or tell me if there is an example that works with big packages and that I can look to?

thanks

[wysota]

Can someone explaining me how to be sure that all the data is received or tell me if there is an example that works with big packages and that I can look to?
You already have that in your code!


if (tcpSocket->bytesAvailable() < nextBlockSize)
{
cout << "Going to break" << endl;
break;
}

The only problem with your code is the "nextBlockSize" variable gets destroyed and loses its value and is initialized with a random value when the method is called again.

[ruben.rodrigues]

I know that I have the code but it doesn't work with big data. The nextBlockSize resets to 0 when I get the readyRead slot triggered and the when I read it again the number is wrong by far. What am I doing wrong there?

[wysota]

You have the nextBlockSize declared as a local variable of the slot connected to the readyRead() signal. The variable should persist across calls (e.g. by making it a member variable of the class).

[ruben.rodrigues]

it already is :(

[wysota]

Then you are overwriting it with bogus values. You can't possibly expect to get several terabytes of data through the socket from the client. It wouldn't even fit into the memory of your computer.