PDA

View Full Version : Text file parsing too slow



Potch
10th June 2008, 23:03
Hello,

I need to parse a tab delimited (or csv) file and display it in a QTableWidget.
So far, I do it this way :


while (!MyStream.atEnd())
{
General_Interface.MainTable->insertRow(Max_Number_of_Lines+1);
buffer_line = MyStream.readLine();
buffer_list = buffer_line.split(Delimiter);

for (int column = 0; column <= Max_Number_of_Columns; column ++)
{
QTableWidgetItem *Cell_Content = new QTableWidgetItem();
Cell_Content->setText(buffer_list[column]);
General_Interface.MainTable->setItem(Max_Number_of_Lines, column,
Cell_Content);

}

Max_Number_of_Lines++;
}

The problem is that the kind of file I have to parse can contains more than 30 columns and 5000 lines, and in these cases it's awfully slow.

How can I optimize this ?
Is it relevant to use a QTextStream as I did ?

Thanks in advance...

wysota
10th June 2008, 23:09
I wouldn't use a text stream in this situation - it only slows everything down without giving any benefits. You can operate on QFile directly.

The problem might be using QTableWidget instead of QTableView. I'd suggest switching to model based approach - this should make things much faster, especially if you first fill the whole model and then set it on the view and not the other way round.

Potch
10th June 2008, 23:33
Thanks for your fast response Wysota.

By
You can operate on QFile directly. do you mean by using QByteArray ?

Concerning the model/view approach, I will rethink my program in this way. For it is the first time I use Qt, item-based approach seemed more friendly to me...
But it seems that the easy way is not always the best !

wysota
11th June 2008, 00:13
do you mean by using QByteArray ?
No, I mean using QIODevice::readLine(). QFile is a QIODevice.

Potch
11th June 2008, 11:44
Thanks for your response. I will go in this direction....

Potch
11th July 2008, 22:16
Hi all,

As advised, I finally used QIODevice::readLine() with a model / view approach, and it works very fast. I easily load a 3 or 4 Mo text file quasi instantaneaous.

For those that it can help :


InputDataModel.setRowCount(0) ;
InputDataModel.setColumnCount(Max_Number_of_Column s + 3) ;

while (InputDataFile.canReadLine())
{
if (Max_Number_of_Lines == 0)
{
buffer_list = buf.split(Delimiter) ;
}

else
{
QByteArray line = InputDataFile.readLine();
QString buffer_line(line);
buffer_list = buffer_line.split(Delimiter);
}

for (int column = 0; column <= Max_Number_of_Columns; column ++)
{
QStandardItem *item = new QStandardItem(buffer_list[column]);
InputDataModel.setItem(Max_Number_of_Lines, column, item) ;
}
}

MytableView->setModel(&InputDataModel);
MytableView->show() ;


Thanks again for your clever advices.....