QNetworkRequest https problem connect missing reply

[Linux5445]

Hello I create this thread because i dont find solution for my problem.

I want to connect in this web site and take cookies: https://cas.univ-ubs.fr/login

my CookieJar class write in a file all cookies of the reply but they have nothing with this website.


void FenPrincipale::takeCookies()
{
QUrl postData;
postData.addQueryItem("username","user101");
postData.addQueryItem("password",var2_Edit->text());
postData.addQueryItem("warn", "true");
postData.addQueryItem("lt", lt);
postData.addQueryItem("_eventId", "submit");
postData.addQueryItem("submit", "SE CONNECTER");

// On crée une requête
QNetworkRequest requete(QUrl(urlEdit1->text()));

requete.setHeader(QNetworkRequest::ContentTypeHead er, "application/x-www-form-urlencoded");
r = m->post(requete, postData.encodedQuery());


// Gestion des erreurs
connect(r, SIGNAL(error(QNetworkReply::NetworkError)), this, SLOT(messageErreur(QNetworkReply::NetworkError)));
// Gestion de fin de téléchargement
connect(r, SIGNAL(finished()), this, SLOT(takeLink()));
}

// The second slot for the reply
void FenPrincipale::takeLink()
{
QNetworkReply *reply = static_cast<QNetworkReply*>(sender());
QNetworkAccessManager *m = reply->manager();
m->setCookieJar(new CookieJar(this));

qDebug() << reply->readAll();
}

Thanks in advance.

[wysota]

So what exactly is the problem? Why are you assigning a cookie jar to the reply? It won't have any effect, it's too late for that.

[Linux5445]

Hello wysota
In the first method takeId i make a GET request to take an id like :c9A188DB9-048B-8F48-0B65-D696212D9AF2_k7D5BBE5A-A72E-33F9-38AD-7C37C4146126

In the takeCookie method i made a cookie in write in my cookie file :
Sun, 12-Aug-2012 17:59:28 GMT; domain=cas.univ-ubs.fr; path=/c9A188DB9-048B-8F48-0B65-D696212D9AF2_k7D5BBE5A-A72E-33F9-38AD-7C37C4146126

next i make a POST request for login with my first cookie, my username, the id, and my password but nothing append.

in firefox when i just go in the web i receveid a cookie but in qt i can't so i simulated the first cookie.

In python a similar code work perfectly then i think i just a little error in my code.


void FenPrincipale::takeId()
{
QNetworkAccessManager *m = new QNetworkAccessManager();
QNetworkRequest requete(QUrl("https://cas.univ-ubs.fr/login"));

QNetworkReply *r = m->get(requete);
qDebug() << "takeId";

connect(r, SIGNAL(finished()), this, SLOT(takeCookies()));
connect(r, SIGNAL(error(QNetworkReply::NetworkError)), this,
SLOT(messageErreur(QNetworkReply::NetworkError)));

}

void FenPrincipale::takeCookies()
{
boutonEnvoyer->setEnabled(false);


QNetworkReply *reply = static_cast<QNetworkReply*>(sender());
QNetworkAccessManager *m = reply->manager();

// Récupère L'id
QString source = reply->readAll();
QString lt = source.section("\"",119,119);
// Récupère le Cookie JSession
source = source.section("\"",41,41);
// Contruit le cookie
QString data;
data = "JSESSIONID";
data += source.remove(QRegExp("[a-z/;]"));
data += "; expires=Sun, 12-Aug-2012 17:59:28 GMT; domain=cas.univ-ubs.fr; path=/";

qDebug() << "takeCookies =" << data;

QFile f("Cookies.txt");
QTextStream out(&f);
if ( f.open(QIODevice::ReadWrite) )
{
QString exist = out.readAll();
if (exist.indexOf("cas") != -1 )
{
out.seek(0);
qDebug() << "seek";
}

out << data;
f.close();
}
m->setCookieJar(new CookieJar(this));

QUrl postData;
postData.addQueryItem("username","");
postData.addQueryItem("password",var2_Edit->text());
postData.addQueryItem("warn", "true");
postData.addQueryItem("lt", lt);
postData.addQueryItem("_eventId", "submit");
postData.addQueryItem("submit", "SE CONNECTER");

// On crée une requête
QNetworkRequest requete(QUrl(urlEdit1->text()));

QNetworkReply *r;
if (box2->isChecked())
{
requete.setHeader(QNetworkRequest::ContentTypeHead er, "application/x-www-form-urlencoded");
r = m->post(requete, postData.encodedQuery());
}

// Gestion des erreurs
connect(r, SIGNAL(error(QNetworkReply::NetworkError)), this, SLOT(messageErreur(QNetworkReply::NetworkError)));
// Gestion de fin de téléchargement
connect(r, SIGNAL(finished()), this, SLOT(takeLink()));

// Gestion de la barre de progression
connect(r, SIGNAL(downloadProgress(qint64, qint64)), this, SLOT(progressionTelechargement(qint64, qint64)));

connect(r, SIGNAL(sslErrors(const QList<QSslError>&)), this, SLOT(ErreurSsl(const QList<QSslError>&)));

connect(m, SIGNAL(authenticationRequired(QNetworkReply*, QAuthenticator*)),
this,SLOT(authentification(QNetworkReply*, QAuthenticator*)));

}

void FenPrincipale::takeLink()
{
QNetworkReply *reply = static_cast<QNetworkReply*>(sender());
QNetworkAccessManager *m = reply->manager();
m->setCookieJar(new CookieJar(this));

qDebug() << reply->readAll();
boutonEnvoyer->setEnabled(true);

QNetworkRequest requete(QUrl("http://web.univ-ubs.fr/edt/ade/custom/myplanning/myPlanning.jsp?top=top.moniframeesup"));
QNetworkReply *r = m->get(requete);
qDebug() << "takeLink";
}

[wysota]

I already told you, you are using the cookie jar incorrectly. Your code is so vague I can't make anything of it. If you tried expressing yourself in a more proper English, maybe I could help you but right now I have no idea what your code is supposed to do. Surely setting a new cookie jar with each request is not the right thing to do. Setting it when you receive a reply is another wrong thing to do. You should set the cookie jar once right after creating the network access manager object. Then you can add cookies which you want the manager to return with requests it makes and you can read cookies from the cookie jar that are returned with replies from the server.

[Linux5445]

I would simply connect to this web site. for her I must make two requestes. A GET request for an id and a POST request with the id, username and password.

I just create the first cookie because i can't received the cookie with the GET request.

I have move the 'setCookiejar' at first.


void FenPrincipale::takeId()
{
QNetworkAccessManager *m = new QNetworkAccessManager();
// Set the cookieJar
m->setCookieJar(new CookieJar(this));
QNetworkRequest requete(QUrl("https://cas.univ-ubs.fr/login"));

// I make a first GET request.
QNetworkReply *r = m->get(requete);
qDebug() << "takeId";

connect(r, SIGNAL(finished()), this, SLOT(takeCookies()));
connect(r, SIGNAL(error(QNetworkReply::NetworkError)), this,
SLOT(messageErreur(QNetworkReply::NetworkError)));

}

void FenPrincipale::takeCookies()
{
QNetworkReply *reply = static_cast<QNetworkReply*>(sender());
QNetworkAccessManager *m = reply->manager();

// I recover the id
QString source = reply->readAll();
QString lt = source.section("\"",119,119);
source = source.section("\"",41,41);

// Build the cookie
QString data;
data = "JSESSIONID";
data += source.remove(QRegExp("[a-z/;]"));
data += "; expires=Sun, 12-Aug-2012 17:59:28 GMT; domain=cas.univ-ubs.fr; path=/";

// Write a cookie in the cookie file
QFile f("Cookies.txt");
QTextStream out(&f);
if ( f.open(QIODevice::ReadWrite) )
{
QString exist = out.readAll();
if (exist.indexOf("cas") != -1 )
{
out.seek(0);
qDebug() << "seek";
}

out << data;
f.close();
}

// Build url contents
QUrl postData;
postData.addQueryItem("username","");
postData.addQueryItem("password",var2_Edit->text());
postData.addQueryItem("warn", "true");
postData.addQueryItem("lt", lt);
postData.addQueryItem("_eventId", "submit");
postData.addQueryItem("submit", "SE CONNECTER");

QNetworkRequest requete(QUrl("https://cas.univ-ubs.fr/login"));
requete.setHeader(QNetworkRequest::ContentTypeHead er, "application/x-www-form-urlencoded");

// I make the second request
QNetworkReply *r = m->post(requete, postData.encodedQuery());


connect(r, SIGNAL(error(QNetworkReply::NetworkError)), this, SLOT(messageErreur(QNetworkReply::NetworkError)));
connect(r, SIGNAL(finished()), this, SLOT(takeLink()));

connect(r, SIGNAL(sslErrors(const QList<QSslError>&)), this, SLOT(ErreurSsl(const QList<QSslError>&)));

connect(m, SIGNAL(authenticationRequired(QNetworkReply*, QAuthenticator*)),
this,SLOT(authentification(QNetworkReply*, QAuthenticator*)));

}

[wysota]

readAll() from the network reply will not return you the cookies. By the time you receive the reply, the cookie should already be in the cookie jar, you don't have to create it yourself. Use QNetworkCookieJar::cookiesForUrl() to retrieve the cookie or dump it somewhere from your implementation of the cookie jar's (as I see you have one) QNetworkCookieJar::setCookiesFromUrl() method.

[Linux5445]

My cookieJar class implementation work good, with any web site in http (google etc...) but with an https request in don't received any cookie. Normally i have to received cookie with https://cas.univ-ubs.fr/login in GET and POST request. I was using readAll for the id, just for that.



// CookieJar class
#include "Cookies.h"

QList<QNetworkCookie> CookieJar::cookiesForUrl ( const QUrl & url ) const
{
return QNetworkCookieJar::cookiesForUrl(url);
}

bool CookieJar::setCookiesFromUrl ( const QList<QNetworkCookie> & cookieList, const QUrl & url )
{
return QNetworkCookieJar::setCookiesFromUrl(cookieList, url);
}

CookieJar::CookieJar(QObject *parent)
:QNetworkCookieJar(parent)
{
QFile cookieFile("Cookies.txt");
if (cookieFile.exists() && cookieFile.open(QIODevice::ReadOnly) )
{
QList<QNetworkCookie> list;
QByteArray line;
while(!(line = cookieFile.readLine()).isNull())
{
list.append(QNetworkCookie::parseCookies(line));
}
setAllCookies(list);
}
}

CookieJar::~CookieJar()
{
QFile cookieFile("Cookies.txt");
if (cookieFile.open(QIODevice::WriteOnly))
{
QTextStream out(&cookieFile);
foreach(const QNetworkCookie & cookie, allCookies())
{
if (!cookie.isSessionCookie())
{
out << cookie.toRawForm() << endl;
qDebug() << "write **********COOKIE******************* ";
}
}
}
}




// My request code
void FenPrincipale::takeId()
{
QNetworkAccessManager *m = new QNetworkAccessManager();
// Set the cookieJar
m->setCookieJar(new CookieJar(this));
QNetworkRequest requete(QUrl("https://cas.univ-ubs.fr/login"));

// I make a first GET request.
QNetworkReply *r = m->get(requete);
qDebug() << "takeId";

connect(r, SIGNAL(finished()), this, SLOT(takeCookies()));
connect(r, SIGNAL(error(QNetworkReply::NetworkError)), this,
SLOT(messageErreur(QNetworkReply::NetworkError)));

}

void FenPrincipale::takeCookies()
{
QNetworkReply *reply = static_cast<QNetworkReply*>(sender());
QNetworkAccessManager *m = reply->manager();

// I recover the id
QString source = reply->readAll();
QString lt = source.section("\"",119,119);
source = source.section("\"",41,41);

// Build url contents
QUrl postData;
postData.addQueryItem("username","");
postData.addQueryItem("password",var2_Edit->text());
postData.addQueryItem("warn", "true");
// I make the id in the URL
postData.addQueryItem("lt", lt);
postData.addQueryItem("_eventId", "submit");
postData.addQueryItem("submit", "SE CONNECTER");

QNetworkRequest requete(QUrl("https://cas.univ-ubs.fr/login"));
requete.setHeader(QNetworkRequest::ContentTypeHead er, "application/x-www-form-urlencoded");

// I make the second request
QNetworkReply *r = m->post(requete, postData.encodedQuery());


connect(r, SIGNAL(error(QNetworkReply::NetworkError)), this, SLOT(messageErreur(QNetworkReply::NetworkError)));
connect(r, SIGNAL(finished()), this, SLOT(takeLink()));

connect(r, SIGNAL(sslErrors(const QList<QSslError>&)), this, SLOT(ErreurSsl(const QList<QSslError>&)));
}

void FenPrincipale::takeLink()
{
QNetworkReply *reply = static_cast<QNetworkReply*>(sender());
QNetworkAccessManager *m = reply->manager();

// ----------> I want to have access to protect file in the web site with my cookie
}

}

[wysota]

Please, understand it - the result returned by readAll() will not contain the part of the conversation with the cookie. Your "lt" and "source" variables will not contain parts of the cookie.

With an ssl connection you don't get anything probably because the site returns an SSL error (handshake failed due to self signed certificate) which you do not handle correctly (see docs for QNetworkReply::sslErrors() signal).

[Linux5445]

I know, I just want to have the id like this in the web site:
<input type="hidden" name="lt" value="_cC02BDB12-D1D9-7155-261D-9D1174D68BD3_k4E4C0CC3-872E-EA91-456C-A29E1E0B4CF5" />

I was using this method for the sslError but i don't have any error:


void FenPrincipale::ErreurSsl(const QList<QSslError> & errors)
{
qDebug () << "errorSSl";
for (int i=0; i < errors.length(); i++)
{
qDebug() << "errorSSl = " << errors[i].errorString();
}
}


Why i don't have any SSL error ? why the web site return nothing (no redirection) ?

[wysota]

I know, I just want to have the id like this in the web site:
<input type="hidden" name="lt" value="_cC02BDB12-D1D9-7155-261D-9D1174D68BD3_k4E4C0CC3-872E-EA91-456C-A29E1E0B4CF5" />
Doing it with QString::section() is a bad idea - if the website contents change it is likely you method will stop working. Better use regular expressions (QRegExp) for getting the id.


I was using this method for the sslError but i don't have any error:
Bearing the fact the certificate of the website is self-signed, you should be getting one error. Does the finished() signal for the network reply even get emitted?

[Linux5445]

Yes is true for the QString section.

When i'm try connected until build a cookie i have a redirection with the cookie in the url.
https://cas.univ-ubs.fr/login;jsessionid=EC0A6DAAB7A739B99B00B17200346C84".
When i don't used the cookie i just have a redirection.

In python i just used (HTTPCookieProcessor)
def __init__(self):
self.CJ = CookieJar()
self.connection = build_opener(HTTPCookieProcessor(self.CJ))

def getConnection(self):
try:
u = self.connection.open("https://cas.univ-ubs.fr/login").read()
f = '<input type="hidden" name="lt" value="'
l = u.rfind(f)
r = u.rfind('" />\n\t\t\t\t\t\t<input type="hidden" name="_eventId"')
data = u[l+len(f):r]
params = urlencode({
"username":self.username,
"password":self.password,
"lt":data,
"_eventId":"submit",
"submit":"SE CONNECTER"
})

self.connection.open("https://cas.univ-ubs.fr/login", params)

I can't translate perfectly in C++ with Qt but i think is possible to work.