PDA

View Full Version : QDataStream::writeRawData() bug?



bmarsden
5th April 2011, 23:46
Hi all -- I've been scouring the forums and Google trying to find more info on this, but have been unsuccessful so here goes (all done on Linux):

I have a client-server setup using QTcpSocket and QTcpServer respectively. In a nutshell, the client and server transmit binary data to one-another by reading/writing data to the QTcpSocket through a QDataStream. The binary data is actually a GNU tar stream -- tar sends the binary stream to a named pipe (FIFO) and I read in the tar stream using QDataStream on the other end of the named pipe. I read the binary in 1MB chunks into a char* from a QFile using read(), write it to the QDataStream using writeRawData(), then write it to the QTcpSocket. On the other side of the connection, I have the readyRead() function return until it has all of the data for that particular chunk buffered (1MBish). Once the block of binary is ready, I read it from the QDataStream using readRawData() (read it into a char*), then write it to a QFile using write(). The QFile is another named pipe that has GNU tar listening on the other end. So, as I write the tar stream to the FIFO, tar reads it and extracts the data.

The problem is, when I build the client/server with Qt 4.7.1 -- there are no issues (...and yes, I am doing a QDataStream::setVersion()) -- I can send data between the client and server and it works beautifully. When I re-compile with Qt 4.5.3, I the binary tar stream received from the server has been corrupt. (or when compiling against 4.7.1 but calling setVersion(QDataStream::Qt_4_5)). I tried reading it with a hex editor and it looks like there are some additional 0's (zeroes) in the tar stream. Is anyone familiar with a bug in QDataStream or maybe a bug with QFile::read() or QFile::write()?

I am at a loss -- and due to platform limitations, I have to run Qt 4.5.3 on one of the platforms we're supporting, so upgrading is not an option. Here are the code snippets -- any help would be appreciated. I have a software release coming up soon and don't know what to do!

This is the server code that reads the binary tar stream from the named pipe and sends it via QTcpSocket to the client. (ignore the char* thisChunk stuff, I thought Qt might be padding some zeroes to binaryChunk if less than 1MB was read)



void Server::sendDataToClient()
{
QByteArray dataBlock;
QFile inFile(exportFifo);
QDataStream outboundStream(&dataBlock, QIODevice::ReadWrite);
QDataStream fileStream(&inFile);

outboundStream.setVersion(QDataStream::Qt_4_5);
fileStream.setVersion(QDataStream::Qt_4_5);

int bytesRead;
int binaryChunkSize = 1048576; // Read 1MB chunks
char* binaryChunk = new char[binaryChunkSize];

qDebug("opening named pipe");

if (!inFile.open(QIODevice::ReadOnly))
{
qDebug("open() on named pipe failed");
logMessage("Unable to open named pipe for reading");
return;
}

bytesRead = inFile.read(binaryChunk, binaryChunkSize);

qDebug() << "read bytes: " << bytesRead;

while(bytesRead > 0)
{
int bytes = bytesRead;
char* thisChunk = new char[bytes];
memcpy(thisChunk, binaryChunk, bytes);

// Create a data block for the binary
outboundStream << quint64(0);
outboundStream << quint16('BD');
outboundStream.writeRawData(thisChunk, bytes);
// outboundStream.writeRawData(binaryChunk, bytesRead);
outboundStream.device()->seek(0);
outboundStream << quint64(dataBlock.size() - sizeof(quint64));

// This is crucial!! When the QDataStream is passed, it retains its
// current reference. This was a PITA BUG TO FIND!!!!!! (for testing)
outboundStream.device()->seek(0);

socket.write(dataBlock);

if(!socket.waitForBytesWritten())
{
// Something bad happened, bail out
logMessage("Timed out waiting for bytes from FIFO");
break;
}

outboundStream.device()->reset();

// Continue to read chunks from the pipe
bytesRead = inFile.read(binaryChunk, binaryChunkSize);
delete[] thisChunk;
qDebug() << "read bytes: " << bytesRead;
}

if(bytesRead == 0)
{
// All bytes have been read from the pipe. Let the client know that
// the transfer is finished.
binaryTransferFinished();
}

// Clean up
inFile.close();
delete[] binaryChunk;
}





And here is the client code that reads the data from the server...




void ComponentNPT::receiveBinaryData(QDataStream& inboundStream, quint64& dataSize)
{
// Don't include the 16-bit response code, we just want the size of the binary
quint64 binarySize = dataSize - sizeof(quint16);

inboundStream.setVersion(QDataStream::Qt_4_5);

int bytesRead = 0;
int tmpRead = 0;
int tmpWrite = 0;
int bytesWritten = 0;
char* binaryChunk = new char[binarySize];

// Loop until we've read all of the binary sent in the data block
while(bytesRead < binarySize)
{
tmpRead = inboundStream.readRawData(binaryChunk, (binarySize - bytesRead));

// tmpRead < 0 is an error, tmpRead == 0 indicates all bytes have been read
if(tmpRead < 0)
qDebug() << "Read error: ";
else if(tmpRead == 0)
break;
else
{
bytesRead += tmpRead;

while(bytesWritten < tmpRead)
{
tmpWrite = outFile.write(binaryChunk, bytesRead);

if(tmpWrite < 0)
{
qDebug() << "Write error: " << outFile.errorString();
break;
}
else
{
bytesWritten += tmpWrite;
}
}
}
}

// Make progress updates based on writes to the FIFO
updateProgressDialog(bytesWritten);

// Clean up
delete[] binaryChunk;
}

JohannesMunk
6th April 2011, 00:39
Hi!

I don't know of any bugs in QFile or QDataStream! Sorry! Can't offer a simple way out :->

I glanced at your code, and couldn't figure out how the size of your last datablock can be correct! As far as I can see you are not releasing your byteArray from one iteration to the next, but use its size. But in the last iteration the stream will not have written till the end but presumably shorter. If my observation is correct, just use the stream->device->position() before you seek back as "datasize". Or much simpler: bytes+10. If not, then its time for me to go to bed, I guess :->

However that might be, I would suggest that you create a small test project, that you can share with us, that leaves the client server stuff out, but reads and writes your files in exactly the same way locally. That should help to pinpoint the problem!

Good luck!

Johannes