[QT4] Counting files in a directory (Linux)

[ucntcme]

First the relevant background: I have a spool directory that I need to get the total number of files in. I don't care about the contents, just how many files. Here's the catch: it's a big number. In this case 1,000,000 files. (Yes it should be hashed but that is not currently an option.)

So, I've tried QProcess with find or ls. They work. I've also tried QDir and it's count() method. It works.

So what's the problem? It's friggin slow. For comparison if I run find manually, piping to "wc -l", it is a lot faster. Doing it by hand takes from 1/4 to 1/2 the time that doing it in Qt using QProcess, and even less than QDir::count() - count takes a few minutes.

So any idea as to which *should* be the fastest route? Why does QDir::count() take *so* long (~4-5 minutes)?

[marcel]

Because it first builds a list of the files and then sorts the list .

...
uint QDir::count() const
{
    Q_D(const QDir);
 
    d->updateFileLists();
    return d->data->files.count();
}
...
inline void QDirPrivate::updateFileLists() const
{
    if(data->listsDirty) {
        QStringList l = data->fileEngine->entryList(data->filters, data->nameFilters);
        sortFileList(data->sort, l, &data->files, &data->fileInfos);
        data->listsDirty = 0;
    }
}

...
uint QDir::count() const
{
    Q_D(const QDir);

    d->updateFileLists();
    return d->data->files.count();
}
...
inline void QDirPrivate::updateFileLists() const
{
    if(data->listsDirty) {
        QStringList l = data->fileEngine->entryList(data->filters, data->nameFilters);
        sortFileList(data->sort, l, &data->files, &data->fileInfos);
        data->listsDirty = 0;
    }
}

To copy to clipboard, switch view to plain text mode 

So, I think you should use platform API, and just count the files, no backstore, or anything similar.

Regards

[ucntcme]

So that explains QDire::count()'s slowness, thanks. That's a mite bit painful though.

But why does QProcess executing find take so much longer than the same command in the shell "manually"? I'd expect a small bit of overhead, but a difference of double or more is a bit unexpected, particularly when the difference is over 30 seconds.

Ah well.
Thanks again.