QFileInfo is very slow

[jho]

Hi,
i have a problem where QFileInfo is very slow.

I have a QStringList with over 15000 paths. I want to get the lastModified date, the fileSize and the fileName. It is very slow. It took about 60-80 seconds.

// collect files and exclude empty mask files
    for(int i=0;i<files3D.count();i++)
    {
        QFileInfo fileInfo = files3D[i];
 
        QDateTime date = fileInfo.lastModified();
        QString str;
        str = date.toString("dd/MM/yyyy hh:mm:ss");
 
            if(fileInfo.size() != 657660)
            {
                fileNames.append(fileInfo.fileName());
                fileDates << str;
                fileSizes << QString::number(fileInfo.size());
            }
 
            qApp->processEvents();
    }

// collect files and exclude empty mask files
    for(int i=0;i<files3D.count();i++)
    {
        QFileInfo fileInfo = files3D[i];

        QDateTime date = fileInfo.lastModified();
        QString str;
        str = date.toString("dd/MM/yyyy hh:mm:ss");

            if(fileInfo.size() != 657660)
            {
                fileNames.append(fileInfo.fileName());
                fileDates << str;
                fileSizes << QString::number(fileInfo.size());
            }

            qApp->processEvents();
    }

To copy to clipboard, switch view to plain text mode 

I think may code is quite simply or have i done something wrong here?
Is there a faster way to get the infos from a file?

[sulliwk06]

For that many paths 60 to 80 seconds doesn't sound entirely unreasonable to me for file actions. Do you have to get the info for all of the files at the same time?

[jho]

Yes I need all of the info at once.

If I compare it with the simple dir command from windows it takes really long.

qt: 80 sec
dir command: 3 sec

My fallback is to use the dir command. But this would be only a workaround.

[wysota]

Your dir command does not process events after reading each entry

[jho]

Hi wysota: I think you mean this line.

qApp->processEvents();

I put this in to keep my application responsive and not freezing. But I already disabled it. So it has nothing to do with the slow reading.

Or do you mean my dir command approach. This would be more of a hack and is not really a good solution.

[wysota]

This code:

#include <QtCore>
 
int main(int argc, char **argv) {
  QDir dir(argv[1]);
  QFileInfoList list = dir.entryInfoList(QDir::NoDotAndDotDot|QDir::Files);
  qDebug() << "File count:" << list.size();
  for(int i=0;i<list.size();++i) {
    const QFileInfo finfo = list.at(i);
    QDateTime mtime = finfo.lastModified();
  }
  return 0;
}

#include <QtCore>

int main(int argc, char **argv) {
  QDir dir(argv[1]);
  QFileInfoList list = dir.entryInfoList(QDir::NoDotAndDotDot|QDir::Files);
  qDebug() << "File count:" << list.size();
  for(int i=0;i<list.size();++i) {
    const QFileInfo finfo = list.at(i);
    QDateTime mtime = finfo.lastModified();
  }
  return 0;
}

To copy to clipboard, switch view to plain text mode 

takes this much time on a directory containing 15000 files:
$ time ./dirtest dir
File count: 15000

real 0m0.095s
user 0m0.074s
sys 0m0.020s

Bottom line: QFileInfo is not slow.

[jho]

I have to look into it again. Maybe with your code although it looks similar to mine. I have to say that the files are stored on a network. But the windows dir command reads it also from the network.

[jho]

I have tested it. It seems to me that using an entry from a QFileInfoList list is way faster than using a QString from a QStringlist with QFileInfo. Very strange...

[jefftee]

QDir::entryInfoList will create all of the QFileInfo objects in one pass of the directory. If you are getting file names into a QStringList and then creating a QFileInfo for each file, then you are accessing the directory multiple times.

Is that what you are referring to? If so, makes sense to me it's way slower, if not, post some code to show what you're referring to.

[jho]

Yes, that is what i meant. And your explanation sounds valid.

Thanks guys for your help.

[wysota]

QDir::entryInfoList will create all of the QFileInfo objects in one pass of the directory. If you are getting file names into a QStringList and then creating a QFileInfo for each file, then you are accessing the directory multiple times.

Is that what you are referring to? If so, makes sense to me it's way slower,

This code:

#include <QtCore>
 
int main(int argc, char **argv) {
  QDir dir(argv[1]);
  QStringList list = dir.entryList(QDir::NoDotAndDotDot|QDir::Files);                
  qDebug() << "File count:" << list.size();    
  for(int i=0;i<list.size();++i) {      
    const QFileInfo finfo = QFileInfo(list.at(i));
    QDateTime mtime = finfo.lastModified();
  }
  return 0;
}

#include <QtCore>

int main(int argc, char **argv) {
  QDir dir(argv[1]);
  QStringList list = dir.entryList(QDir::NoDotAndDotDot|QDir::Files);                
  qDebug() << "File count:" << list.size();    
  for(int i=0;i<list.size();++i) {      
    const QFileInfo finfo = QFileInfo(list.at(i));
    QDateTime mtime = finfo.lastModified();
  }
  return 0;
}

To copy to clipboard, switch view to plain text mode 

takes this much (different computer, don't compare with previous results):
$ time ./dirtest dir
File count: 15000

real 0m0.068s
user 0m0.055s
sys 0m0.012s

I don't see any significant differences.

[jefftee]

I don't see any significant differences.

I stand corrected. Looks like the 2nd test was actually a little faster than the first, go figure!