Recursive directory scan with abort function in a non blocking manner

[Lykurg]

Hi,

this is the task: Scan a folder recursive for - lets say - png files and give their addresses back in a QStringList. Furthermore the count of the found images should be displayed in a label or whatever during the scan. And of course the gui is not allowed to freeze and you should be able to abort the scan.

And the solution is ... ? Or better asked, what is the best, easiest and best-performance solution?

Right now I have a simple subclasses thread and do like:

class ThreadReadDir : public QThread
{
Q_OBJECT
public:
explicit ThreadReadDir(QObject* parent = 0)
: QThread(parent)
{
m_stopped = false;
}

void stop()
{
m_stopped = true;
}

void readDir(QDir dir)
{
m_dir = dir;
if (!m_stopped)
start();
}

void run()
{
readDir_private(m_dir);
}

QStringList files() const
{
return m_files;
}

Q_SIGNALS:
void currentFileCountChanged(int);

private:
QStringList m_files;
QDir m_dir;
bool m_stopped;

void readDir_private(QDir dir)
{
if (m_stopped)
return;
QStringList files = dir.entryList(QStringList() << "*.png", QDir::Files, QDir::Name);
for (int i = 0; i < files.count(); ++i)
m_files << dir.absolutePath() + QDir::separator() + files.value(i);
Q_EMIT currentFileCountChanged(m_files.count());

if (m_stopped)
return;
QStringList directories = dir.entryList(QStringList(), QDir::AllDirs | QDir::NoDotAndDotDot, QDir::Name);
for (int i = 0; i < directories.count(); ++i)
{
if (m_stopped)
return;
readDir_private(dir.absolutePath() + QDir::separator() + directories.at(i));
}
}
};With
// Constructor
ThreadReadDir* m_filethread = new ThreadReadDir(this);
QObject::connect(m_filethread, SIGNAL(currentFileCountChanged(int)), this, SLOT(setFileCount(int)));

// After scan request
m_filethread->readDir(QDir(/*...*/));

//
void XXX::setFileCount(int cnt)
{
ui->statRead->setText(QString("%1 Files").arg(cnt));
}

//
void XXX::OnAbortRequest()
{
m_filethread->stop();
m_filethread->wait();
}

It works, but it is a hell of work for such a simple request and therefore I wonder if there is a easier and better to maintain way to achieve that. QtConcurrent::run would be fine if one could cancel it.


Thanks for sharing your thoughts,

Lykurg

[franz]

The QtConcurrent::map*() functions are cancellable through the QFuture object they return. Maybe it's possible to mock something up using that? (it would probably be a little hackish though :))

[MTK358]

Not sure if this is relavent, but the file extension is NOT a certain way to tell the type of a file. For example i use Linux, and file extensions have no meaning to it. The only real reason for them is for the user to tell the file type easily, and because *some* programs pay attention to them.

Unlike in Windows, in Linux I can call a PNG file "image.jpg" and it will work just fine, because *nix and most *nix programs look at the contents of the file, not the extension.


michael@michael-desktop ~ $ file image.jpg
image.jpg: JPEG image data, JFIF standard 1.01
michael@michael-desktop ~ $ mv image.jpg image.png
michael@michael-desktop ~ $ file image.png
image.png: JPEG image data, JFIF standard 1.01

[hobbyist]

I'd probably go with QThread as well. Something like


class Scanner : public QThread
{
Q_OBJECT

public:
explicit Scanner(const QDir &dir, QObject *parent = 0)
: iterator(dir, QDirIterator::Subdirectories), QThread(parent), stopped(false)
{
}

void run()
{
while(!stopped && iterator.hasNext()) {
QString entry(iterator.next());
if(QFileInfo(entry).suffix().toLower() == "png") {
fileList << entry;
emit fileCountChanged(fileList.count());
}
}
}

QStringList files() const
{
return fileList;
}

public slots:
void stop()
{
stopped = true;
}

signals:
void fileCountChanged(int);

private:
QDirIterator iterator;
QStringList fileList;
bool stopped;
};

Then simply start(), stop() and wait(), as in the original code snippet above.

[Lykurg]

Oh, I love Qt! And I should probably look more often in the docs, also for working solutions since I wasn't aware of QDirIterator. This will save me doing the recursive function. Nice! Thanks for pointing that out. (One small improvement if somebody want use it later: since the iterator uses a QFileInfo internal better use that (QDirIterator::fileInfo) instead of creating a second one.)

@franz: the map* functions are nice but personal I find them hard to maintain (especial in that case). A capsuled Thread is also easier to reuse.
@MTK358: The suffixes are of course meaningless. If you want be sure that it is really a png or any other file you have to do "deeper" tests.


And one follow up: Did you normally subclass QThread or use QObject::moveToThread()? (I am just curious how others did it.)

[MTK358]

The suffixes are of course meaningless. If you want be sure that it is really a png or any other file you have to do "deeper" tests.

Yes, sure. If I would make a simple example I would just have it search for *.png, too. Anyway, if it matters, in Wikipedia it says that a PNG file should always begin with:


89 50 4E 47 0D 0A 1A 0A