A simple HTTP server going wrong.

[spraff]

Hello, and thanks for reading. Perhaps my issues are more about the HTTP protocol than Qt, but here goes...

The code at the bottom is my simple prototype HTTP server for a web interface for my app. qWarning ouputs the lengths of the headers it receives and its POST contents, raw and interpreted. It responds to GET requests with a little form containing two fields, that much works fine.

I thought this would be a matter of reading headers, followed by reading arbitrary data. Apparently not, since the behaviour is very inconsistent.

For orientation, here's the qWarning output when the page is used in Firefox, submitting nothing in input1 and the string "x" in input2 (some headers omitted for clarity)

POST:20
Content-Type::49
Content-Length::20
Blank
16 bytes
Headers done
 
input1=&input2=x
input1 =>
input2 => x

POST:20
Content-Type::49
Content-Length::20
Blank
16 bytes
Headers done

input1=&input2=x
input1 =>
input2 => x

To copy to clipboard, switch view to plain text mode 

All present and correct.

In Konqueror, however, the GET page works fine but hitting the post button causes a "Connection to host is broken" message. Despite this, the program outputs this (some headers omitted again, differences are included):

POST:20
Pragma::18
Cache-control::25
Content-Type::49
Headers done
 
 
 =>

POST:20
Pragma::18
Cache-control::25
Content-Type::49
Headers done


 =>

To copy to clipboard, switch view to plain text mode 

Note that there is a Content-Type header but not a Content-length. What's going on here? Notice that no blank header-ending line is present either.

Even more confusing is behaviour in Opera. If input1 has contents, everything works fine. If input2 has contents but input1 does not then it very often sets the Content-Length header properly but when the "content" variable (see code) is read, there is nothing there. Sometimes, rarely, arbitrarily, the data is there.

Here's the code for this little headache. Am I using the Qt objects incorrectly or am I misunderstanding the HTTP protocol?

To summarise:

Why does Konqueror say connection failed for POST requests, even though the code executes?

Why does the "content" variable not always contain data, even when Content-Length is correctly set?

And finally, why does ~HTTPRequest never get called even though the socket closes and the socket is its parent object? (I tried calling delete explicitly but that broke everything?)

HTTPInterface :: HTTPInterface (QString pass, unsigned int port, QObject * p)
: QTcpServer (p) {
	password = pass;
	if (!pass.size()) {_ok=false;return;}
 
	connect(this,SIGNAL(newConnection()),
		this,SLOT(incoming()) );
 
	_ok = listen (QHostAddress::Any, port);
}
 
bool HTTPInterface :: ok () const {return _ok;}
 
void HTTPInterface :: incoming () {
	while (QTcpSocket * connection = nextPendingConnection()) {
		new HTTPRequest (connection);
	}
}
 
HTTPRequest :: HTTPRequest (QTcpSocket * s) : QObject (s), socket (s) {
	connect(s,SIGNAL(readyRead()),
		this,SLOT(respond()) );
}
 
HTTPRequest :: ~HTTPRequest () {
	qWarning ("~HTTPRequest");
}
 
namespace {
	QString title = "Remote Administration";
	QString page = "http://foo.com";
	QString heading =
		"<h1>"+title+"</h1>\n"
		"<h2><a href=\"" + page + "\">" + page + "</a></h2>\n";
	QString stylesheet =
		"\n<style type=\"text/css\">\n"
		"h1 {\n"
		"\tcolor: red;\n"
		"}\n"
		"</style>\n\n";
 
	QRegExp header_space ("[\\s\\n\\r]+");
	QRegExp anything ("[^\\s\\n\\r]");
	QRegExp post_var ("([^=]+)=([^&]*)");
};
 
void HTTPRequest :: respond () {
	QTextStream out (socket);
 
	while (socket->canReadLine()) {
		QString l = socket -> readLine ();
 
		// Headers end with a blank line.
		if (anything.indexIn(l) < 0) {
			qWarning("Blank");
			// If there's non-header content, read it.
			if (headers.contains("Content-Length:")) {
				l = headers["Content-Length:"][1];
				content = socket -> read(l.toLong());
				qWarning(QString("%1 bytes").arg(l.toLong())
						.toAscii());
			}
			continue;
		}
 
		// Save each header, indexed by the "FOO:" prefix.
		QStringList tokens = l.split (header_space);
		foreach (QString t, tokens) {
			headers[tokens[0]] = tokens;
		}
		qWarning(QString("%1:%2")
				.arg(tokens[0]).arg(l.size()).toAscii() );
	}
	qWarning("Headers done\n");
 
	out <<	"HTTP/1.0 200 Ok\r\n"
		"Content-Type: text/html; charset=\"utf-8\"\r\n"
		"\r\n"
		"<html><head><title>" << title << "</title></head>\n"
		"<body>" << stylesheet << heading;
 
	// Interpret POST.
	qWarning(content);
	QStringList post_vars = QString(content).split('&');
	foreach (QString pair, post_vars) {
		QString key = pair.section('=',0,0);
		QString val = pair.section('=',1);
		qWarning(QString("%1 => %2").arg(key).arg(val).toAscii());
		key = QUrl::fromPercentEncoding (key.toAscii());
		val = QUrl::fromPercentEncoding (val.toAscii());
		POST[key] = val;
	}
 
	if (headers.contains("POST")) {
		out << "<h3>POST</h3><table><tbody>\n";
		for (QMapIterator<QString,QString> i(POST); i.hasNext(); ) {
			i.next();
			out << "\t<tr><td>" << i.key()
			     << "</td><td>" << i.value() << "</td></tr>\n";
		}
		out << "</tbody></table>\n";
	}
 
	else { // Everything else defaults to GET.
		out << "<p><form method=\"post\" action=\"act\">\n"
			"\t<input type=\"text\" name=\"input1\" />\n"
			"\t<input type=\"text\" name=\"input2\" />\n"
			"\t<input type=\"submit\" /></form>\n</p>\n";
	}
 
	socket->close();
}

HTTPInterface :: HTTPInterface (QString pass, unsigned int port, QObject * p)
: QTcpServer (p) {
	password = pass;
	if (!pass.size()) {_ok=false;return;}

	connect(this,SIGNAL(newConnection()),
		this,SLOT(incoming()) );

	_ok = listen (QHostAddress::Any, port);
}

bool HTTPInterface :: ok () const {return _ok;}

void HTTPInterface :: incoming () {
	while (QTcpSocket * connection = nextPendingConnection()) {
		new HTTPRequest (connection);
	}
}

HTTPRequest :: HTTPRequest (QTcpSocket * s) : QObject (s), socket (s) {
	connect(s,SIGNAL(readyRead()),
		this,SLOT(respond()) );
}

HTTPRequest :: ~HTTPRequest () {
	qWarning ("~HTTPRequest");
}

namespace {
	QString title = "Remote Administration";
	QString page = "http://foo.com";
	QString heading =
		"<h1>"+title+"</h1>\n"
		"<h2><a href=\"" + page + "\">" + page + "</a></h2>\n";
	QString stylesheet =
		"\n<style type=\"text/css\">\n"
		"h1 {\n"
		"\tcolor: red;\n"
		"}\n"
		"</style>\n\n";

	QRegExp header_space ("[\\s\\n\\r]+");
	QRegExp anything ("[^\\s\\n\\r]");
	QRegExp post_var ("([^=]+)=([^&]*)");
};

void HTTPRequest :: respond () {
	QTextStream out (socket);

	while (socket->canReadLine()) {
		QString l = socket -> readLine ();

		// Headers end with a blank line.
		if (anything.indexIn(l) < 0) {
			qWarning("Blank");
			// If there's non-header content, read it.
			if (headers.contains("Content-Length:")) {
				l = headers["Content-Length:"][1];
				content = socket -> read(l.toLong());
				qWarning(QString("%1 bytes").arg(l.toLong())
						.toAscii());
			}
			continue;
		}

		// Save each header, indexed by the "FOO:" prefix.
		QStringList tokens = l.split (header_space);
		foreach (QString t, tokens) {
			headers[tokens[0]] = tokens;
		}
		qWarning(QString("%1:%2")
				.arg(tokens[0]).arg(l.size()).toAscii() );
	}
	qWarning("Headers done\n");

	out <<	"HTTP/1.0 200 Ok\r\n"
		"Content-Type: text/html; charset=\"utf-8\"\r\n"
		"\r\n"
		"<html><head><title>" << title << "</title></head>\n"
		"<body>" << stylesheet << heading;

	// Interpret POST.
	qWarning(content);
	QStringList post_vars = QString(content).split('&');
	foreach (QString pair, post_vars) {
		QString key = pair.section('=',0,0);
		QString val = pair.section('=',1);
		qWarning(QString("%1 => %2").arg(key).arg(val).toAscii());
		key = QUrl::fromPercentEncoding (key.toAscii());
		val = QUrl::fromPercentEncoding (val.toAscii());
		POST[key] = val;
	}

	if (headers.contains("POST")) {
		out << "<h3>POST</h3><table><tbody>\n";
		for (QMapIterator<QString,QString> i(POST); i.hasNext(); ) {
			i.next();
			out << "\t<tr><td>" << i.key()
			     << "</td><td>" << i.value() << "</td></tr>\n";
		}
		out << "</tbody></table>\n";
	}

	else { // Everything else defaults to GET.
		out << "<p><form method=\"post\" action=\"act\">\n"
			"\t<input type=\"text\" name=\"input1\" />\n"
			"\t<input type=\"text\" name=\"input2\" />\n"
			"\t<input type=\"submit\" /></form>\n</p>\n";
	}

	socket->close();
}

To copy to clipboard, switch view to plain text mode 

[caduel]

Just to be on the safe side: tcp connections may deliver data in several packets. canReadLine()==false does not mean that all the data you sent has been received and processed. It just means that the data already available (in full lines) has been processed.
Make sure you are prepared for your header to arrive in multiple pieces (i.e. do not start processing an incomplete header).

HTH