QNetworkAccessManager and http redirection

[grayfox]

Is it true that QNetworkAccessManager has no built-in support for http redirection?
I would like to download raw html from the Internet but it seems that I can't get to the final redirected webpage.



#ifndef PROGRAM_H
#define PROGRAM_H

#include <QtNetwork>
#include <QtCore>

class Program : public QObject
{
Q_OBJECT

public:
Program()
{
net = 0;
}
~Program()
{
delete net;
}
void run()
{
net = new QNetworkAccessManager;
connect(net, SIGNAL(finished(QNetworkReply*)),this, SLOT(replyFinished(QNetworkReply*)));
net->get(QNetworkRequest(QUrl("http://www.yahoo.com")));
}

private slots:
void replyFinished(QNetworkReply* r){
QString s = r->readAll();
qDebug() << s;
}
private :
QNetworkAccessManager* net;
};

#endif // PROGRAM_H



When run it called, I got the following:

<!-- w84.fp.sk1.yahoo.com uncompressed/chunked Thu Jul 7 08:37:01 PDT 2011 -->
instead of the final redirected webpage.

[squidge]

What you are getting is the last line only. I get much more:

--- snip! ---
if ( "undefined" != typeof(rt_LogTime) ) {
rt_LogTime(
"t1", rtJSLoad - rtJSStart, true);
rt_LogTime("t2", rtJSDone - rtJSLoad
, true);
if ("undefined" != typeof(rtAdStart) && "undefined" != typeof(rt
AdDone)) { rt_LogTime("t3", rtAdDone - rtAdStart, true); }
}
</script></body>

</html>
<!-- bid=701 -->
<!-- sid=2023538075 -->
2c

<!-- myproperty:myservice-us:0:Success -->

50
<!-- w25.fp.ch1.yahoo.com uncompressed/chunked Thu Jul 7 14:33:30 PDT 2011 -->

[ChrisW67]

What greyfox is getting is the body content of the initial server response. It is not following the redirect, which you will find in the Location: header of the response.

In the finished() signal you could check the QNetworkRequest::RedirectionTargetAttribute attribute and, if the return is valid, create a new request and get() it. You might want to test for redirect loops or endless redirects though.

[squidge]

Although I didn't follow any redirect requests, and to confirm it I also got the above by using telnet to connect to www.yahoo.com, port 80.

[ChrisW67]

I get a full response through telnet with a bare request also. However, using wget or QNetworkAccessManager the "Host:" header is sent and that triggers a different response from Yahoo.


chrisw@newton /tmp $ telnet www.yahoo.com 80
Trying 98.137.149.56...
Connected to any-fp.wa1.b.yahoo.com.
Escape character is '^]'.
GET / HTTP/1.0
Host: www.yahoo.com

HTTP/1.0 302 Found
Date: Fri, 08 Jul 2011 07:41:09 GMT
P3P: policyref="http://info.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE LOC GOV"
Cache-Control: private
Set-Cookie: IU=deleted; expires=Thu, 08-Jul-2010 07:41:08 GMT; path=/; domain=.yahoo.com
Set-Cookie: fpc=d=mE9W1x3vmwH5qQfykKO9i17Mi68As827SzEoRYZfeHl. 7YKK16lgEMZn2AeBTCkH6rMVH2R7O3CIDu1vDGA9rMySC_7rhu _x1X_RXpUZsQFrhS_3gLr7VfouTwfBLAwisocjqWRT..pSmkbn _0UfXPfN1SxzCgHt9l5Sq8l4dyX7Cx21tKKImt_2DUe36ZK5cM j63JY-&v=2; expires=Sat, 07-Jul-2012 07:41:09 GMT; path=/; domain=www.yahoo.com
Location: http://au.yahoo.com/?p=us
Vary: Accept-Encoding
Content-Type: text/html; charset=utf-8
Age: 0
Server: YTS/1.20.0

<!-- w27.fp.sp2.yahoo.com uncompressed/chunked Fri Jul 8 00:41:09 PDT 2011 -->
Connection closed by foreign host.

This is a redirect, in my case to the Australian subsidiary. The only content in the response is the one line the OP is seeing.

[squidge]

Indeed. I forgot that QNetworkAccessManager sent the 'host' header.