Concurrent file downloading

[Alir3z4]

Download one file at the time with source code below is easy, and also download multiply file also isn't problem when using QQueue, but downloading files in parallel i can't get it!
Assume we have:
I make code shorter and remove other codes for demonstration purpose ;)
downloader.h

#ifndef DOWNLOADER_H
#define DOWNLOADER_H

#include <QObject>
#include <QFile>
#include <QFileInfo>
#include <QNetworkAccessManager>
#include <QNetworkRequest>
#include <QNetworkReply>
#include <QUrl>

class Dowloader : public QObject
{
Q_OBJECT
public:
explicit Dowloader(QObject *parent = 0);

public slots:
void theDownload(const QString &urlLink);

private slots:
void startRequest(const QUrl &url);
void httpReadyRead();
void httpFinished();

private:
QNetworkAccessManager manager;
QNetworkReply *reply;
QFile *file;
};

#endif // DOWNLOADER_H

downloader.cpp

#include "dowloader.h"

Dowloader::Dowloader(QObject *parent) :
QObject(parent)
{}

void Dowloader::theDownload(const QString &urlLink)
{
QUrl url(urlLink);
QFileInfo fInfo(url.path());
QString fileName = fInfo.fileName();
if(fileName.isEmpty())
fileName = "download";

file = new QFile(fileName);

if(!file->open(QIODevice::WriteOnly)){
delete file;
file = 0;
return;
}
startRequest(url);
}

void Dowloader::replyMetaDataChanged()
{}

void Dowloader::startRequest(const QUrl &url)
{
QNetworkRequest request(url);
reply = manager.get(request);
// conncet signals
connect(reply, SIGNAL(metaDataChanged()), this, SLOT(replyMetaDataChanged()));
connect(reply, SIGNAL(readyRead()), this, SLOT(httpReadyRead()));
connect(reply, SIGNAL(finished()), this, SLOT(httpFinished()));
}

void Dowloader::httpReadyRead()
{
if(file) file->write(reply->readAll());
}

void Dowloader::httpFinished()
{
file->flush();
file->close();
reply->deleteLater();
reply = 0;
delete file;
file = 0;
}
The main problem here is, in the httpReadyRead() ans httpFinished() how can operate on the correct file, reply? as you see in the *.h file they are *pointer, it means when i add new file current QNetworkReply *reply, QFile *file, gonna re-initialize with new values!
What changes need in this code to make Concurrent file downloading possible? // Doesn't mean provide some code or something, just give me a scenario about it!

Thanks

[Lykurg]

QSignalMapper and QHash can help.

[Alir3z4]

QSignalMapper and QHash can help.
Yes, first thing came to my head was using QHash, QSignalMapper, but what should i keep in the QHash?
pointer or reference?
pointer like this?

QHash<QUrl, QHash<QNetworkReply*, QFile*> > *downloads;
if pointers, what happen when i add new download and re-initialize *reply*, *file, or something like that? // or maybe i shouldn't use *reply*, *file and just make pointers add them to hash on the fly?
or i have to keep ref in it?

[Lykurg]

Pointers and file/replay aren't getting overwritten since you store it in the hash. Use QHash<QNetworkReply*, QFile*>. Just make sure you delete the pointers after the downloads are finished.

[ChrisW67]

You can also not store the reply pointer at all and use the QNetworkAccessManager::finished() signal, which passes the QNetworkReply object that finished. This is really only suitable for smaller files where the entire content is easily stored in memory and progress indication is not important.

[Alir3z4]

You can also not store the reply pointer at all and use the QNetworkAccessManager::finished() signal, which passes the QNetworkReply object that finished. This is really only suitable for smaller files where the entire content is easily stored in memory and progress indication is not important.
As you said, for small file file.
//
By the way, i'm doing the QSignalMapper approach, but the new problem [weird] is just at the first time readyRead() signal emitted httpReadyRead() gonna work properly
startRequest edited:


void Dowloader::startRequest(const QUrl &url)
{
QNetworkRequest request(url);
reply = manager.get(request);
// conncet signals
downloads->insert(reply, file);
qDebug() << "downloads->insert(" << reply << "," << file << ");";

connect(downloads->key(file), SIGNAL(readyRead()), signalMapper, SLOT(map()));
signalMapper->setMapping(downloads->key(file), downloads->value(reply));
connect(signalMapper, SIGNAL(mapped(QObject*)), this, SLOT(httpReadyRead(QObject*)));
connect(signalMapper, SIGNAL(mapped(QObject*)), this, SLOT(httpFinished(QObject*)));
}


i changed the httpReadyRead to

void Dowloader::httpReadyRead(QObject *_file)
{
QFile *theFile = qobject_cast<QFile*>(_file);
if(file) theFile->write(downloads->key(theFile)->readAll());
}

and downloads also is:

QHash<QNetworkReply*, QFile*> *downloads

i change the code as i learn from docs, i don't know why writing to file in httpReadyRead works just at th first time, but next times when readyRead emitted from reply, theFile->openMode() print-out "NotOpen"
any idea?

Added after 30 minutes:

EDITED:
Before reading the reply data [reply->readAll();] if i force the *theFile to reopen it again [ theFile->open(QIODevice::WriteOnly); ] then i can writing the file
But as soon readyRead emitted the openMode of theFile changed to NotOpen!
What is the problem ?

[Alir3z4]

Actually i found the problem why *file openMode() changed to "NotOpen", this is QSignalMapper issue and get me confuse!
this is the guilty line:

connect(signalMapper, SIGNAL(mapped(QObject*)), this, SLOT(httpFinished(QObject*)));
The problem is httpFinished(QObject*) gonna emitted right after httpReadyRead(QObject*) emits !
when i // comment the httpFinished(QObject*), everything is fine! because in the httpFinished(QObject*) i shoulda flush()/close()/0/delete the file pointer, and of course do deleteLater()/remove form hash the reply pointer!

So why httpFinished(QObject*) emitted immediately even when reply actually is not finished! ?

Added after 8 minutes:

Or should i use different QSignlaMapper for httpfinished() ?

What? actually yes, i used different QSignalMapper and it works!
did i get something wrong about QSignalMapper?

[mentalmushroom]

look how you established your connections: you connected two slots to the same signal, that is why your slots are called together.

[Alir3z4]

look how you established your connections: you connected two slots to the same signal, that is why your slots are called together.
ignore initialed connecting signals code
i connected them before in this way:

void Dowloader::startRequest(const QUrl &url)
{
QNetworkRequest request(url);
reply = manager.get(request);
QHash<QNetworkReply*, QFile*>::iterator i = downloads->insert( reply, file);
downloads->insert(reply, file);
signalMapper->setMapping(i.key(), i.value());
connect(i.key(), SIGNAL(readyRead()), signalMapper, SLOT(map()));
connect(i.key(), SIGNAL(finished()), signalMapper, SLOT(map()));

connect(signalMapper, SIGNAL(mapped(QObject*)), this, SLOT(httpReadyRead(QObject*)));
connect(signalMapper, SIGNAL(mapped(QObject*)), this, SLOT(httpFinished(QObject*)));
}

is this wrong ?

[mentalmushroom]

it is wrong unless you want httpFinished(QObject*) to be called when readyRead is emitted

[Alir3z4]

it is wrong unless you want httpFinished(QObject*) to be called when readyRead is emitted

Ow, i really lost in it now, how i want to called when readyRead is emitted ?
I connect them as usual :| ?, i did't found any way to do it in different way!

[mentalmushroom]

connect(i.key(), SIGNAL(readyRead()), signalMapper, SLOT(map()));
connect(i.key(), SIGNAL(finished()), signalMapper, SLOT(map()));

connect(signalMapper, SIGNAL(mapped(QObject*)), this, SLOT(httpReadyRead(QObject*)));
connect(signalMapper, SIGNAL(mapped(QObject*)), this, SLOT(httpFinished(QObject*)));


think a little bit how it works. when readyRead is emitted it calls map(), the same is done when finished is emitted. i guess, map slot emits mapped signal. you have two slots connected to the same signal (mapped), therefore they both will be invoked when any of the signals is emitted.

[Alir3z4]

connect(i.key(), SIGNAL(readyRead()), signalMapper, SLOT(map()));
connect(i.key(), SIGNAL(finished()), signalMapper, SLOT(map()));

connect(signalMapper, SIGNAL(mapped(QObject*)), this, SLOT(httpReadyRead(QObject*)));
connect(signalMapper, SIGNAL(mapped(QObject*)), this, SLOT(httpFinished(QObject*)));


think a little bit how it works. when readyRead is emitted it calls map(), the same is done when finished is emitted. i guess, map slot emits mapped signal. you have two slots connected to the same signal (mapped), therefore they both will be invoked when any of the signals is emitted.

Aha you are right, but i got your point, but i think using one instance of QSignalMapper in this situation is wrong!
If i'm wrong correct me:i can use QSignalMapper for 1 SLOT which is associated to many signals, right ? // as i see the doc example about it!

[mentalmushroom]

Yes, I think, you should use a different SignalMapper for the second signal.

Technically you can connect several slots to one signal, also the same slot can be connected to several different signals, finally, you can connect the same slot to the same signal several times. Only you think whether this is what you want, because, I think, in your case you don't need this.

[Alir3z4]

Yes, I think, you should use a different SignalMapper for the second signal.

That's what i posted http://www.qtcentre.org/threads/47540-Concurrent-file-downloading?p=214502#post214502
And you really assure me about it ;)
Thanks