Large File Processing

[sankar]

Hello,
I would like to know which is the optimized way to append large file content to a small file. I did the following, but it doesn't work for more than a 1 GB file, it throws error, while reading file content during "QByteArray myFileData = fileHandle2.readAll();"

QFile fileHandle, fileHandle2;
	fileHandle.setFileName("SourceFile.txt");
	fileHandle2.setFileName("VectorFile.txt");
	if(fileHandle.open(QIODevice::Append|QIODevice::Text))
	{
		if(fileHandle2.open(QIODevice::ReadOnly|QIODevice::Text))
		{
			QByteArray myFileData = fileHandle2.readAll();
 
			if(myFileData.isEmpty())
				QMessageBox::critical(this,"File Operations","File read error...");
 
			if(fileHandle.write(myFileData) == -1)
				QMessageBox::critical(this,"File Operations","File append error...");
		}
	}
 
	fileHandle.close();
	fileHandle2.close();

QFile fileHandle, fileHandle2;
	fileHandle.setFileName("SourceFile.txt");
	fileHandle2.setFileName("VectorFile.txt");
	if(fileHandle.open(QIODevice::Append|QIODevice::Text))
	{
		if(fileHandle2.open(QIODevice::ReadOnly|QIODevice::Text))
		{
			QByteArray myFileData = fileHandle2.readAll();

			if(myFileData.isEmpty())
				QMessageBox::critical(this,"File Operations","File read error...");

			if(fileHandle.write(myFileData) == -1)
				QMessageBox::critical(this,"File Operations","File append error...");
		}
	}

	fileHandle.close();
	fileHandle2.close();

To copy to clipboard, switch view to plain text mode 

Regards,
Sankar.

[sonulohani]

QString QTextStream::readAll ()

Reads the entire content of the stream, and returns it as a QString. Avoid this function when working on large files, as it will consume a significant amount of memory.

Calling readLine() is better if you do not know how much data is available.

[ChrisW67]

Use the QIODevice::read() and QIODevice::write() functions with a buffer of known size in a while loop.