How to make a proxy server?

[Passerby]

Hi, I'm Passerby, a newbie to Qt.

I've been trying to make a local proxy server for web browsers (HTTP protocol; something like BFilter (http://bfilter.sourceforge.net)) with Qt.
I thought the QNetworkAccessManager would be the key class, but it doesn't seem to be able to handle requests from other places. So I checked the whole QtNetwork module, and found that only QLocalServer and QTcpServer provide a listen() method. But they are quite different from QNetworkReply or QHttp. They are also not able to set a fixed local port to listen to.

So which classes should I use, in order to receive HTTP requests from a specified port, and deal with that like a proxy server?

Thanks.

[auba]

Mmmh... help says:
QTcpServer::bool listen ( const QHostAddress & address = QHostAddress::Any, quint16 port = 0 )

so the port can be set. My servers are all QTcpServers, works fine.

[Passerby]

Yes OK QTcpServer::listen() supports port#, but I'm trying to build a proxy server, not a game/IM/etc server.
So is QTcpServer the only choice, and I'll have to extract the information from the TCP connection and reassemble the HTTP request all by myself?

[Passerby]

Hi, it's me again.

So now I'm using QTcpServer. For every new connection, I use a QHttp to send request, and write the response back to the socket (please tell me if this is stupid; I just thought it would be easier to apply filter).

Everything works fine, except for the Transfer-Encoding: chunked response. All browsers will immediately close the socket after one chunk and act as "page can not be displayed".

I googled and wiki-ed the definition of chunked encoding, did some test, and finally found out that what QHttp::readAll() return is no longer chunked.

Then I tried removing the Transfer-Encoding header before writing. That works, but leads to another problem: by the definition of chunked encoding, the message-sender is allowed to "append" headers after the message body (so-called "trailer"). So if I remove the Transfer-Encoding header, the trailer will be lost (or treated as improper message, depending on how QHttp handle; I haven't tested that yet).

I can re-chunk the message by "repair"ing the size-line of each chunk, but still, I'll have to determine where's the trailer part and avoid counting their length. Since the size-line is already removed by QHttp, I can't think of anything except the colon and space as the identifier of trailer. But there's always chance that the message body contains a colon and space.

So can anyone please tell me how to deal with the chunked encoding transfer? Many thanks.

[Fred]

Hi Passerby,

this is a very interesting project since the development of BFilter is discontinued for over two years now. Do you have developed your program further? I have tried to program a proxy but I don't found out how to correctly to handle post requests of bigger files from the browser.

Greetings

[tbscope]

For any questions about the protocols you handle in the proxy, check the rfc's
http://www.w3.org/Protocols/rfc2616/rfc2616.html

[giusepped]

Me too, I am developing a proxy in Qt.
It works, but not for all website.
Part of the source code can be found here

http://stackoverflow.com/questions/7009259/qtcpsocket-stucks-sometimes-for-dynamic-created-java-script

Check for example this video webserver 87.22.235.24.
It hangs...
Regards

[Fred]

Hi giusepped,

have you solved the problem? Is the source code available so I can test it, maybe I can help. I'm looking forward for an alternative for bfilter since the project is unfortunately discontinued.

regards

[giusepped]

Hi giusepped,

have you solved the problem? Is the source code available so I can test it, maybe I can help. I'm looking forward for an alternative for bfilter since the project is unfortunately discontinued.

regards

I solved by doing myself a header parser, and an unzipper for gzipped sites. But do not work always. I am in trouble for just one site. If you can help, I can share the code with you. Please, reply in that case by email or pm.
Regards