After finishing my project I decide to investigate what could have been causing the slow response in Qt SQL classes.
Looking at QSqlQuery code
https://qt.gitorious.org/qt/qtbase/s.../qsqlquery.cpp line #905, the method record() seems to be the problem.
{
if (isValid()) {
for (int i = 0; i < rec.count(); ++i)
rec.setValue(i, value(i));
}
return rec;
}
QSqlRecord QSqlQuery::record() const
{
QSqlRecord rec = d->sqlResult->record();
if (isValid()) {
for (int i = 0; i < rec.count(); ++i)
rec.setValue(i, value(i));
}
return rec;
}
To copy to clipboard, switch view to plain text mode
As you can see the code loops through an internal record structure and populates a QSqlRecord which is returned by the method. Putting a call to record() inside any loop, now doesn't look like a good idea.
I made a project to test calling record() in diferent ways. And exporting data with 300K records I get results varying from 03:00 to 00:30 using three different implementations.
Project: QIBaseExport.zip
Database DDL: employee.sql.bz2
I coundn't attach the data file because it's size exceeds the Qt Centre limitation. But this database it's based in MySQL open database sample.
All data and table definition could be downloaded from https://launchpad.net/test-db/
Anyway, before anything I should have read Qt documentation as it says from the record method:
Returns a QSqlRecord containing the field information for the current query. If the query points to a valid row (isValid() returns true),
the record is populated with the row's values. An empty record is returned when there is no active query (isActive() returns false).
To retrieve values from a query, value() should be used since its index-based lookup is faster.
In the following example, a SELECT * FROM query is executed. Since the order of the columns is not defined, QSqlRecord::indexOf() is used to obtain the index of a column.
and yet provides an example of how it should be used
qDebug() << "Number of columns: " << rec.count();
int nameCol = rec.indexOf("name"); // index of the field "name"
while (q.next())
qDebug() << q.value(nameCol).toString(); // output all names
QSqlQuery q("select * from employees");
QSqlRecord rec = q.record();
qDebug() << "Number of columns: " << rec.count();
int nameCol = rec.indexOf("name"); // index of the field "name"
while (q.next())
qDebug() << q.value(nameCol).toString(); // output all names
To copy to clipboard, switch view to plain text mode
note that the call to record is done only once and outside to while loop.
Bookmarks