(Solved) QNetworkAccessManager and http-stream without headers

[muehle.kai]

Hello Folks,

I'am using QT version 4.6.3 under debian 6.0.3. (Compiling using qmake and make, gcc version 4.4.5 (Debian 4.4.5-8))

To read from a measuring system a data stream using http and some additional data from html-pages i tried to use QNetworkAccessManager via GET (QNetworkRequest) and QNetworkReply.

This runs flawless with the html-Pages.

reply = qnam.get(QNetworkRequest(url));
connect(reply, SIGNAL(error(QNetworkReply::NetworkError)),
          this, SLOT(httpError(QNetworkReply::NetworkError)));
connect(reply, SIGNAL(readyRead()),
          this, SLOT(httpReadyRead()));
connect(reply, SIGNAL(finished()),
          this, SLOT(httpFinished()));

reply = qnam.get(QNetworkRequest(url));
connect(reply, SIGNAL(error(QNetworkReply::NetworkError)),
          this, SLOT(httpError(QNetworkReply::NetworkError)));
connect(reply, SIGNAL(readyRead()),
          this, SLOT(httpReadyRead()));
connect(reply, SIGNAL(finished()),
          this, SLOT(httpFinished()));

To copy to clipboard, switch view to plain text mode 

Trying to read the stream the readyRead-routine will not run. To figure out what happened I also connected the error-routine. The reported error is error #2: RemoteHostClosedError.

However, WGET can read and save the stream:

wget -S HOST:PORT
--2012-04-27 13:36:46--  HOST:PORT
Trying to connec to HOST:PORT... connected.
HTTP-Request sent, waiting for reply...200 No Headers, assuming HTTP/0.9. 
Length: not specified
Saving to »index.html«

wget -S HOST:PORT
--2012-04-27 13:36:46--  HOST:PORT
Trying to connec to HOST:PORT... connected.
HTTP-Request sent, waiting for reply...200 No Headers, assuming HTTP/0.9. 
Length: not specified
Saving to »index.html«

To copy to clipboard, switch view to plain text mode 

Firefox also can save the stream.

A longer google-search found no appropriate answers. Only some ssl-related stuff, what does not relate to my problem, as far as I can see. What I have found in QT-Sources (4.7.0, in 4.6.0 it was not there, if it is in 4.6.3 I do not know): Inside qhttpnetworkconnectionchannel.cpp ( see arrows --> in code):

case QAbstractSocket::RemoteHostClosedError:
        // try to reconnect/resend before sending an error.
        // while "Reading" the _q_disconnected() will handle this.
        if (state != QHttpNetworkConnectionChannel::IdleState && state != QHttpNetworkConnectionChannel::ReadingState) {
            if (reconnectAttempts-- > 0) {
                closeAndResendCurrentRequest();
                return;
            } else {
                errorCode = QNetworkReply::RemoteHostClosedError;
            }
        } else if (state == QHttpNetworkConnectionChannel::ReadingState) {
            if (!reply->d_func()->expectContent()) {
                // No content expected, this is a valid way to have the connection closed by the server
                return;
            }
            if (reply->contentLength() == -1 && !reply->d_func()->isChunked()) {
-->            // There was no content-length header and it's not chunked encoding,
-->            // so this is a valid way to have the connection closed by the server
                return;
            }
            // ok, we got a disconnect even though we did not expect it
            errorCode = QNetworkReply::RemoteHostClosedError;
        } else {
            errorCode = QNetworkReply::RemoteHostClosedError;
        }
        break;

case QAbstractSocket::RemoteHostClosedError:
        // try to reconnect/resend before sending an error.
        // while "Reading" the _q_disconnected() will handle this.
        if (state != QHttpNetworkConnectionChannel::IdleState && state != QHttpNetworkConnectionChannel::ReadingState) {
            if (reconnectAttempts-- > 0) {
                closeAndResendCurrentRequest();
                return;
            } else {
                errorCode = QNetworkReply::RemoteHostClosedError;
            }
        } else if (state == QHttpNetworkConnectionChannel::ReadingState) {
            if (!reply->d_func()->expectContent()) {
                // No content expected, this is a valid way to have the connection closed by the server
                return;
            }
            if (reply->contentLength() == -1 && !reply->d_func()->isChunked()) {
-->            // There was no content-length header and it's not chunked encoding,
-->            // so this is a valid way to have the connection closed by the server
                return;
            }
            // ok, we got a disconnect even though we did not expect it
            errorCode = QNetworkReply::RemoteHostClosedError;
        } else {
            errorCode = QNetworkReply::RemoteHostClosedError;
        }
        break;

To copy to clipboard, switch view to plain text mode 

Does that mean, that QNAM believes that an answer without headers and without chunking is a HostClosed situation and ends the connection itself?

I'am at a loss at the moment, because I have developed my source code with via WGET temporarily stored data. My local webserver certainly added headers and length. Can someone give some deeper insights into QNAM, why that is happening?

I'am working around the problem for now, wget-ing packages of the stream and feeding to my program but that is a bad hack because I loose temporal coherence on the data.

Cheers,
Kai

[ChrisW67]

An HTTP request was made and a non-HTTP response was received. This is a misconfigured HTTP server or not an HTTP server at all.

If you run your wget with the "-S" option do you see any headers? What is the URL?

[muehle.kai]

Hi Chris,

An HTTP request was made and a non-HTTP response was received. This is a misconfigured HTTP server or not an HTTP server at all.

If you run your wget with the "-S" option do you see any headers? What is the URL?

The stream comes from a measuring system (running linux), which is inside it's own subnet without access to the internet. Data rate approx. 30MB/s. I have only limited access to the system. But your suggestion may be right, that there is no http server at all, and the system spits out only data at that particular port. Here is what wget -S said, translated from german :-):

wget -S http://192.168.1.100:5432
--2012-05-02 09:35:05--  http://192.168.1.100:5432/
Trying to connect to 192.168.1.100:5432... connected.
HTTP-Request sent, waiting for reply... 
Length: not specified
Saving to »index.html«

wget -S http://192.168.1.100:5432
--2012-05-02 09:35:05--  http://192.168.1.100:5432/
Trying to connect to 192.168.1.100:5432... connected.
HTTP-Request sent, waiting for reply... 
Length: not specified
Saving to »index.html«

To copy to clipboard, switch view to plain text mode 

What could I do to convince QNAM to read that stream without headers? If that cannot be achieved, what do you suggest to read those kind of streams?

Cheers,
Kai

[ChrisW67]

QTcpSocket is how you would connect to an arbitrary TCP endpoint and just receive data as it is sent.

[muehle.kai]

Hello Chris,

QTcpSocket is how you would connect to an arbitrary TCP endpoint and just receive data as it is sent.

that's it. I additionally implemented a QTcpSocket and did some minor changes on the sources. Now I can read via QNetworkAccessManager (already saved of data) or directly from the stream via QTcpSocket. Thank you for the suggestion!

Cheers,
Kai