trying to make QHttp synchronous or threaded

[ber_44]

My task is to download a file and then process it right away. Of course it does not work with QHttp's asynchronous API.

I tried to make QHttp synchronous by calling QHttp::setSocket() and then applying the blocking approach (see Blocking Fortune Client Example), but since requests in QHttp are queued, QHttp::setHost() does not guarantee that the socket has even started connecting, so QAbstractSocket::waitForConnected() does not work.

Now my second option is to put QHttp in a QThread and then call QThread::wait() -- but I would need a good example for that, having no experience with multi-threaded programming.

[marcel]

Don't complicate everything with threads.

QHttp emits the signals requestStarted( int id ) and requestFinished( int id, bool error ).
With these signals you can synchronize with QHttp.

Also you can use the state() function and stateChanged() signal.

For example:

CMyClass::CMyClass( QWidget* parent )
:QWidget( parent )
{
    mHttp = new QHttp( this );
    mSetHostReqID = mHttp->setHost( "www.somesite.org" );
 
    connect( mHttp, SIGNAL( requestStarted( int ) ), this, SLOT ( httpStartedRequest( int ) ) );
    connect( mHttp, SIGNAL( requestFinished( int, bool ), this, SLOT( httpFinishedRequest ( int, bool ) ) );
    connect( mHttp, SIGNAL( stateChanged( int ) ), this, SLOT( httpChangedState( int ) ) );
}
 
void CMyClass::httpStartedRequest( int reqId )
{
    if( reqId == mSetHostReqID )
    {
         // Code for this request goes here
    }
    else if( reqId == someOtherRequest)
   {
         //Code for some other request 
   }
}
 
void CMyClass::httpFinishedRequest( int reqId, bool error )
{
    if( reqId == mSetHostReqID )
    {
         if( error )
             // Do something if error occured
         else
             // Code for this finifhed request goes here
    }
    else if( reqId == someOtherRequest)
   {
         if( error )
             // Do something if error occured
         else
             //Code for some other request finished
   }
}
 
void CMyClass::httpChangedState( int newState )
{
     //Code to do something when this state occurs
     switch( newState )
     {
     case QHttp::Unconnected:
         //Code to handle this state
     break;
     case QHttp::HostLookup:
         //Code to handle this state
      break;
     case QHttp::Connecting:
         //Code to handle this state
      break;
     case QHttp::Sending:
         //Code to handle this state
      break;
     case QHttp::Reading:
         //Code to handle this state
      break;
     case QHttp::Connected:
         //Code to handle this state
      break;
     case QHttp::Cosing:
         //Code to handle this state
      break;
     }
}

CMyClass::CMyClass( QWidget* parent )
:QWidget( parent )
{
    mHttp = new QHttp( this );
    mSetHostReqID = mHttp->setHost( "www.somesite.org" );

    connect( mHttp, SIGNAL( requestStarted( int ) ), this, SLOT ( httpStartedRequest( int ) ) );
    connect( mHttp, SIGNAL( requestFinished( int, bool ), this, SLOT( httpFinishedRequest ( int, bool ) ) );
    connect( mHttp, SIGNAL( stateChanged( int ) ), this, SLOT( httpChangedState( int ) ) );
}

void CMyClass::httpStartedRequest( int reqId )
{
    if( reqId == mSetHostReqID )
    {
         // Code for this request goes here
    }
    else if( reqId == someOtherRequest)
   {
         //Code for some other request 
   }
}

void CMyClass::httpFinishedRequest( int reqId, bool error )
{
    if( reqId == mSetHostReqID )
    {
         if( error )
             // Do something if error occured
         else
             // Code for this finifhed request goes here
    }
    else if( reqId == someOtherRequest)
   {
         if( error )
             // Do something if error occured
         else
             //Code for some other request finished
   }
}

void CMyClass::httpChangedState( int newState )
{
     //Code to do something when this state occurs
     switch( newState )
     {
     case QHttp::Unconnected:
         //Code to handle this state
     break;
     case QHttp::HostLookup:
         //Code to handle this state
      break;
     case QHttp::Connecting:
         //Code to handle this state
      break;
     case QHttp::Sending:
         //Code to handle this state
      break;
     case QHttp::Reading:
         //Code to handle this state
      break;
     case QHttp::Connected:
         //Code to handle this state
      break;
     case QHttp::Cosing:
         //Code to handle this state
      break;
     }
}

To copy to clipboard, switch view to plain text mode 

Try it. You shouldn't need a thread at all.

Regards.

[ber_44]

Do you mean this:

c::c {
connect(http, SIGNAL(requestFinished(int, bool)),
this, SLOT(httpRequestFinished(int, bool)));
// ...
}

void c::f() {
// ...
reqId[GETNBYTES] = http->get(dir, file);
// program continues in c::httpRequestFinished()
}

void c::httpRequestFinished(int reqId_, bool error) {
switch (reqId_) {
case reqId[GETNBYTES]:
if (error)
handle_it();
while (1) {
// ...
reqId[i] = http->get(dir, file); // --> I can't imagine these to be not queued
}
// ...
}
}

Looks like that requests in a loop are incompatible with this approach.

[marcel]

You don't need that while. QHttp is async, therefore the signals.

You start downloading the first file as you said, then in httpRequestFinished you download the next file, and so on... You always have to wait for the current file to be downloaded, therefore you need to put the files waiting for download in a queue.

[ber_44]

Actually, I used while() just to download a number of files. But now I realized that it is not a problem.
However, now I'm getting a SIGSEGV right at the end of httpRequestFinished().
I wonder why.

[marcel]

Most likely it crashes because of something you do, not because of the signal/slot.
Could help you if you post some code...

Regards.

[marcel]

What is GETNBYTES ( why do you store requests like this? ) ?
What does handle_it?

[ber_44]

Thanks for offering your help again.
This is my code that gets SIGSEGV at "case SET_HOST: case SET_USER: break;":

#include "httpwindow.h"
 
extern CReadImagesDatabase d;
extern QTimer *timer;
 
HttpWindow::HttpWindow(QWidget *parent, const char * Dir, const char * Dir2, const char *Url, const char *Subdir)
 : QWidget(parent) {
/* typical values:
dir (const char *): ....../images_data/
dir2 (const char *): ......./images_data/tmp/
url (QUrl): [url]http://localhost:8000/[/url]
subdir / subdir of images.csv and nbytes.txt on the server / (QString): ./
*/
 
    our_parent = parent;
    dir = Dir;
    dir2 = Dir2;
    subdir = Subdir;
 
    http = new QHttp;
    url = new QUrl(QString(Url), QUrl::TolerantMode);
    user_abort = false;
    httpRequestAborted = false;
    reqId.insert(http->setHost(url->host(), url->port() != -1 ? url->port() : 80), SET_HOST);
    if (!url->userName().isEmpty())   
        reqId.insert(http->setUser(url->userName(), url->password()), SET_USER);
 
    progressBar = new QProgressBar;
    progressBar->setMinimumWidth(progressBar->fontMetrics().width(QLatin1String("X")) * MAX_WIDTH);
    statusLabel = new QLabel;
    quitButton = new QPushButton(tr("&Quit"));
 
    connect(http, SIGNAL(requestFinished(int, bool)),
            this, SLOT(httpRequestFinished(int, bool)));
    connect(http, SIGNAL(responseHeaderReceived(const QHttpResponseHeader &)),
            this, SLOT(readResponseHeader(const QHttpResponseHeader &)));
    connect(quitButton, SIGNAL(clicked()), 
            this, SLOT(cancelDownload()));
 
    QVBoxLayout *mainLayout = new QVBoxLayout;
    mainLayout->addWidget(statusLabel);
    mainLayout->addWidget(progressBar);
    mainLayout->addWidget(quitButton, 0, Qt::AlignRight);
    setLayout(mainLayout);
 
    setWindowTitle(tr("Updating Isis Images"));
 
}
 
void HttpWindow::updateArchive() {
 
    note(0, tr("compare byte count of .csv files"));
 
    file = new QFile(dir2 + QString("nbytes.txt"));
    if (!file->open(QIODevice::ReadWrite)) {
        // no error message, since it is OK to work from a read-only media
        delete file; file = 0;
        return; 
    }
 
    user_abort = false;
    httpRequestAborted = false;
    reqId.insert(http->get(url->path() + subdir, file), GET_N_BYTES);
}
 
void HttpWindow::httpRequestFinished(int reqId_, bool error) {   
    if (user_abort) {
        if (file) { file->remove(); delete file; file = 0; }
        myQuit(); 
    }
 
    qint64 i, tmpi = 0;
    bool index_in_tmp;
    QList<QString>::iterator j;
    QList<QString> new_list;
    CReadImagesDatabase e;
 
    switch (reqId.value(reqId_)) { 
 
    case SET_HOST: case SET_USER: 
    break; 
    case GET_N_BYTES:
           // ...
    break;
    case GET_NEW_INDEX:
           // ...
    break;
    default: // 0, 1, ... - get fixed image; GET_N_BYTES, etc. are negative values
       if (reqId.value(reqId_) >= 0) 
          // ...
       else 
          // ...
    } // end of switch
} 
 
void HttpWindow::myQuit() {
// ...
}
 
void HttpWindow::cancelDownload() {
// ...
}
 
void HttpWindow::readResponseHeader(const QHttpResponseHeader &responseHeader) {
// ...
}
 
void HttpWindow::updateDataReadProgress(int bytesRead, int totalBytes) {
// ...
}
 
void HttpWindow::note(QString s, QString t) {
// ...
}

#include "httpwindow.h"

extern CReadImagesDatabase d;
extern QTimer *timer;

HttpWindow::HttpWindow(QWidget *parent, const char * Dir, const char * Dir2, const char *Url, const char *Subdir)
 : QWidget(parent) {
/* typical values:
dir (const char *): ....../images_data/
dir2 (const char *): ......./images_data/tmp/
url (QUrl): [url]http://localhost:8000/[/url]
subdir / subdir of images.csv and nbytes.txt on the server / (QString): ./
*/

    our_parent = parent;
    dir = Dir;
    dir2 = Dir2;
    subdir = Subdir;

    http = new QHttp;
    url = new QUrl(QString(Url), QUrl::TolerantMode);
    user_abort = false;
    httpRequestAborted = false;
    reqId.insert(http->setHost(url->host(), url->port() != -1 ? url->port() : 80), SET_HOST);
    if (!url->userName().isEmpty())   
        reqId.insert(http->setUser(url->userName(), url->password()), SET_USER);

    progressBar = new QProgressBar;
    progressBar->setMinimumWidth(progressBar->fontMetrics().width(QLatin1String("X")) * MAX_WIDTH);
    statusLabel = new QLabel;
    quitButton = new QPushButton(tr("&Quit"));

    connect(http, SIGNAL(requestFinished(int, bool)),
            this, SLOT(httpRequestFinished(int, bool)));
    connect(http, SIGNAL(responseHeaderReceived(const QHttpResponseHeader &)),
            this, SLOT(readResponseHeader(const QHttpResponseHeader &)));
    connect(quitButton, SIGNAL(clicked()), 
            this, SLOT(cancelDownload()));

    QVBoxLayout *mainLayout = new QVBoxLayout;
    mainLayout->addWidget(statusLabel);
    mainLayout->addWidget(progressBar);
    mainLayout->addWidget(quitButton, 0, Qt::AlignRight);
    setLayout(mainLayout);

    setWindowTitle(tr("Updating Isis Images"));

}

void HttpWindow::updateArchive() {

    note(0, tr("compare byte count of .csv files"));

    file = new QFile(dir2 + QString("nbytes.txt"));
    if (!file->open(QIODevice::ReadWrite)) {
        // no error message, since it is OK to work from a read-only media
        delete file; file = 0;
        return; 
    }

    user_abort = false;
    httpRequestAborted = false;
    reqId.insert(http->get(url->path() + subdir, file), GET_N_BYTES);
}

void HttpWindow::httpRequestFinished(int reqId_, bool error) {   
    if (user_abort) {
        if (file) { file->remove(); delete file; file = 0; }
        myQuit(); 
    }

    qint64 i, tmpi = 0;
    bool index_in_tmp;
    QList<QString>::iterator j;
    QList<QString> new_list;
    CReadImagesDatabase e;

    switch (reqId.value(reqId_)) { 

    case SET_HOST: case SET_USER: 
    break; 
    case GET_N_BYTES:
           // ...
    break;
    case GET_NEW_INDEX:
           // ...
    break;
    default: // 0, 1, ... - get fixed image; GET_N_BYTES, etc. are negative values
       if (reqId.value(reqId_) >= 0) 
          // ...
       else 
          // ...
    } // end of switch
} 

void HttpWindow::myQuit() {
// ...
}

void HttpWindow::cancelDownload() {
// ...
}

void HttpWindow::readResponseHeader(const QHttpResponseHeader &responseHeader) {
// ...
}

void HttpWindow::updateDataReadProgress(int bytesRead, int totalBytes) {
// ...
}

void HttpWindow::note(QString s, QString t) {
// ...
}

To copy to clipboard, switch view to plain text mode 

and the header:

#ifndef HTTPWINDOW_H
#define HTTPWINDOW_H
 
#include <QtGui>
#include <QtNetwork>
#include <QtAlgorithms>
#include <stdio.h>
#include <stdlib.h>
#include "ReadImagesDatabase.h"
 
// these consts must not be >= 0
#define SET_HOST -1
#define SET_USER -2
#define GET_N_BYTES -3
#define GET_NEW_INDEX -4
 
#define MAX_WIDTH 30   
#define DEBUG
 
class HttpWindow : public QWidget
{
    Q_OBJECT
public:
    HttpWindow(QWidget *parent, const char *Dir, const char *Dir2, const char *Url, const char *Subdir);
public slots:
    void updateArchive();
private slots:
    void cancelDownload();
    void httpRequestFinished(int requestId, bool error);
    void readResponseHeader(const QHttpResponseHeader &responseHeader);
    void updateDataReadProgress(int bytesRead, int totalBytes);
private:
    void note(QString, QString);
    QWidget *our_parent;
    bool user_abort;
    QString subdir;
    void downloadFile(QString);
    void myQuit();
    QProgressBar *progressBar;
    QPushButton *quitButton;
    QLabel *statusLabel;
    QHttp *http;
    QFile *file, *file1;
    QUrl *url;
    const char *dir, *dir2;
    QMap<int, int> reqId;
    bool httpRequestAborted;
};
 
#endif

#ifndef HTTPWINDOW_H
#define HTTPWINDOW_H

#include <QtGui>
#include <QtNetwork>
#include <QtAlgorithms>
#include <stdio.h>
#include <stdlib.h>
#include "ReadImagesDatabase.h"

// these consts must not be >= 0
#define SET_HOST -1
#define SET_USER -2
#define GET_N_BYTES -3
#define GET_NEW_INDEX -4

#define MAX_WIDTH 30   
#define DEBUG

class HttpWindow : public QWidget
{
    Q_OBJECT
public:
    HttpWindow(QWidget *parent, const char *Dir, const char *Dir2, const char *Url, const char *Subdir);
public slots:
    void updateArchive();
private slots:
    void cancelDownload();
    void httpRequestFinished(int requestId, bool error);
    void readResponseHeader(const QHttpResponseHeader &responseHeader);
    void updateDataReadProgress(int bytesRead, int totalBytes);
private:
    void note(QString, QString);
    QWidget *our_parent;
    bool user_abort;
    QString subdir;
    void downloadFile(QString);
    void myQuit();
    QProgressBar *progressBar;
    QPushButton *quitButton;
    QLabel *statusLabel;
    QHttp *http;
    QFile *file, *file1;
    QUrl *url;
    const char *dir, *dir2;
    QMap<int, int> reqId;
    bool httpRequestAborted;
};

#endif

To copy to clipboard, switch view to plain text mode 

[wysota]

Is this more or less what you're trying to achieve?

http://www.qtcentre.org/forum/f-qt-p...-qt4-5695.html

Especially you might want to take a look at the bundle attached here.

[ber_44]

No, wysota, that was a different problem. He wanted to handle multiple requests at the same time, while I want to have one request at a time and in a blocking fashion -- the event loop should wait until the request is finished.
As I said, QAbstractSocket::waitFor*() functions do not work. marcel suggested me to use QHttp::requestFinished() instead of QThread::wait(), but it currently crashes my program. See the code is my previous post.

[wysota]

Did you have a look at the attachment? IMHO it works exactly the same way you want your code to work and it uses requestFinished()

If you want to force "synchronousness", use a busy loop:

while(!someFlagActive) qApp->processEvents();

while(!someFlagActive) qApp->processEvents();

To copy to clipboard, switch view to plain text mode 

And make sure the flag is being set when QHttp finishes its job (in a slot connected to done() or requestFinished()).

Your code probably segfaults because requestFinished() is emitted after every http "command" - including QHttp::setHost() so probably "reqId.value(reqId_)" returns an invalid value.

[marcel]

Look carefully at the QMap you are using.
It makes no sense crashing in the first to cases where you do nothing...
Are you sure you don't use those negative error indices somewhere else?

[marcel]

Instead of QMap::value( const Key& ), try using QMap::value( const Key&, const &T ).
This way you'll be able to provide a default (known) value in case the key does not map to any value. You can then use this value to avoid such crashes.

[wysota]

How about using QMap::contains() or QMap::find() and checking the iterator for equality with end() (better performance than with contains())?

[wysota]

By the way... This:

while(!someFlagActive) qApp->processEvents();

while(!someFlagActive) qApp->processEvents();

To copy to clipboard, switch view to plain text mode 

is equivalent to this (just designed worse :P):

QHttp *http;
//...
QEventLoop loop;
connect(http, SIGNAL(done(bool)), &loop, SLOT(quit())); // you can use a different signal here
loop.exec(QEventLoop::AllEvents|QEventLoop::WaitForMoreEvents);
doSomethingAfterDoneIsEmitted();

QHttp *http;
//...
QEventLoop loop;
connect(http, SIGNAL(done(bool)), &loop, SLOT(quit())); // you can use a different signal here
loop.exec(QEventLoop::AllEvents|QEventLoop::WaitForMoreEvents);
doSomethingAfterDoneIsEmitted();

To copy to clipboard, switch view to plain text mode 

Remember that done() will be processed before the loop exits!

[ber_44]

> probably "reqId.value(reqId_)" returns an invalid value

No, I printed it, it's just fine. The crash was caused by something else... Now I'm just getting
free(): invalid pointer 0x40016ca0!
after returning from the SET_HOST / SET_USER case (request is queued to happen after HttpWindow::HttpWindow()

But this does not bother me too much. My program goes on to the first download, BUT when I print file->bytesAvailable() in the GET_N_BYTES case it is zero-size. However, the file does get downloaded just fine, as I see it after the program finishes.

> If you want to force "synchronousness", use a busy loop
> QEventLoop loop;
> connect(http, SIGNAL(done(bool)), &loop, SLOT(quit())); // you can use a different signal here
> loop.exec(QEventLoop::AllEvents|QEventLoop::WaitFo rMoreEvents);
> doSomethingAfterDoneIsEmitted();

That seems to be the solution. IMHO we just replaced the original idea of using QThread::wait() with a local event loop (the program won't execute the next statement until that local event loop has finished).