PDA

View Full Version : Problem by subclassing the QIODevice



erqsor
29th December 2011, 00:32
Hello,

We can read this in the Qt Documentation (in the QIODevice class description):

By subclassing QIODevice, you can provide the same interface to your own I/O devices. Subclasses of QIODevice are only required to implement the protected readData() and writeData() functions. QIODevice uses these functions to implement all its convenience functions, such as getChar(), readLine() and write(). QIODevice also handles access control for you, so you can safely assume that the device is opened in write mode if writeData() is called.

So I tried to subclass QAbstractSocket by reimplementing readData and writeData, but I can't read any data.
I tried readAll, getChar and reading data with a QDataStream and that didn't work.
When I reimplement readAll and getChar by using my readData, that works fine.
So do you think that reimplementing readData and writeData is really sufficient ? Have I to reimplement other functions ?

erqsor
13th January 2012, 11:34
A lil up ? :)

wysota
13th January 2012, 12:25
It is sufficient for reimplementing QIODevice, not for reimplementing a subclass of QIODevice which itself customizes the behaviour of a couple of methods from QIODevice.

erqsor
13th January 2012, 12:44
OK, Thanks, that's interesting.
Do you have an idea of the functions I have to reimplement ?

I begin by implementing the common ways to read data from a QAbsctratSocket.

wysota
13th January 2012, 13:46
Why are you subclassing QAbstractSocket and not QIODevice? What is it exactly that you want to achieve?

erqsor
13th January 2012, 15:03
I created a websocket server lib and I'm trying to improve the genericity of my lib.
http://gitorious.org/qtwebsocket

wysota
13th January 2012, 22:09
Aren't websockets based on TCP?

erqsor
14th January 2012, 15:14
That's true, so I'm using QTcpServer and QTcpSockets in background.
But websockets are over the TCP. So I created a QWsServer and a QWsSocket class.
I created a new signal QWsSocket::frameReceived to get the data, but I want to use the traditional socket signals (readyRead) and functions (read, readData, stream...).
I started it, but as I said, I'm surprised that the Qt documentation say we just have to reimplement the writeData and readData functions.

wysota
14th January 2012, 15:40
That's true, so I'm using QTcpServer and QTcpSockets in background.
But websockets are over the TCP. So I created a QWsServer and a QWsSocket class.
Why don't you just subclass QTcpSocket?


I started it, but as I said, I'm surprised that the Qt documentation say we just have to reimplement the writeData and readData functions.
As I said, the docs say so about QIODevice, not QAbstractSocket. The latter reimplements 8 of its superclass methods changing their default behaviour.

erqsor
14th January 2012, 16:18
I started by subclassing QTcpSocket. But I had to change this, because how can you use the readyRead signal to read incoming data through the TCP socket and next emiting the same signal to allow QWsSocket users to get the websocket incoming data.
There is an architectural problem :(

wysota
14th January 2012, 17:36
Isn't websocket data related to the tcp data you receive? I don't understand why do you want to mimic QAbstractSocket so much. Either you are dealing with low-level mechanism in which case what QTcpSocket offers is just enough or you want to implement a high-level mechanism (with access to all the headers and stuff) in which case you should probably just implement a backend for QNetworkAccessManager. WebSockets is an application layer concept and QAbstractSocket is a transport layer concept.

erqsor
16th January 2012, 15:35
Websockets is a transport layer too. You have to specify data type (binary, text, ...) and frame formats, size, mask, etc...
As I readed, QNetworkAccessManager can be used to send requests and receive answers, how can I etablish a socket connexion like this ?
In my case, I'm the server, I have requests to send, but not only, I have mainly requests to receive and treat.
Thats the aim of the websocket mechanism. Find a alternative to ajax polling by using sockets.

I don't know why do you want I change my architecture ? I just needed some informations on the QIODevice subclassing...

wysota
16th January 2012, 15:44
Websockets is a transport layer too. You have to specify data type (binary, text, ...) and frame formats, size, mask, etc...
So it's not a transport layer.


As I readed, QNetworkAccessManager can be used to send requests and receive answers, how can I etablish a socket connexion like this ?
By reimplementing QNetworkAccessManager::createRequest() and making the connection to the server.


I don't know why do you want I change my architecture ? I just needed some informations on the QIODevice subclassing...
I don't want to change your architecture. You are the one complaining about your current architecture. If you ask me, you can even derive your socket from QRect if you really like to do additional work.

erqsor
16th January 2012, 15:52
OK, I thought that was a transport layer.

I'm not closed to any suggestion, but here I don't understand how you can etablish the connexion.
In my case, the C++ application is on the server side, and the javascript web application is on the client side, and this web application send the first request.
So the first thing I have to do in the C++ application is to receive a request, and not send requests. How can you do that with QNetworkAccessManager::createRequest ?
Or do you have a turorial/example ?

wysota
16th January 2012, 19:21
OK, I thought that was a transport layer.
If I understand how WebSockets work, they make a request over HTTP which is above the transport layer (TCP serves as the transport layer).


I'm not closed to any suggestion, but here I don't understand how you can etablish the connexion.
Qt is open source, take a look there :)


In my case, the C++ application is on the server side, and the javascript web application is on the client side, and this web application send the first request.
Then I have totally no idea why you want to have a subclass of QAbstractSocket. You should have something similar to QTcpServer which is not derived from either QIODevice or QTcpSocket.

What you are implementing is a regular HTTP server serving regular HTTP requests. I don't know what your web application does but unless it tries to do arbitrary streaming over websocket API (which is doubtful) there is really no point in deriving anything from QAbstractSocket.

erqsor
16th January 2012, 22:35
If I understand how WebSockets work, they make a request over HTTP which is above the transport layer (TCP serves as the transport layer).
Yes, the client side is Google Chrome or Mozilla Firefox.
This client send the first HTTP request.
So by my side (server side), I have to get a client request (to etablish a connexion). And up to now, I used sockets to reply other requests.

You can read the ref here : http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-17

Thats not "my" application. I'm making a generic API (webserver socket with Qt). So users can do simple requests or streaming, etc... (what they want...).
So for now, I give to my API users a QWsSocket with which they can communicate with different clients.

Added after 13 minutes:

I looked this documentation (In French, sorry) http://qt.developpez.com/doc/4.7/qnetworkaccessmanager/
And they speak about requests sended by the server to receive replies.
But here I'm doing the opposite. So if you want, I could qnetworkaccessmanager if I had to developpe a client application, but here I need to develop a server application. Thats not the server which requests the clients, but clients which requests the server.

wysota
16th January 2012, 23:08
I'm making a generic API (webserver socket with Qt). So users can do simple requests or streaming, etc... (what they want...).
So for now, I give to my API users a QWsSocket with which they can communicate with different clients.
The thing is there is no benefit in returning a subclass of QAbstractSocket. Of course it is technically possible to do that however in my personal opinion it is better to deliver an object more similar to QNetworkReply --- something that contains the headers of the request and means to read the body of the request through QIODevice API.


I looked this documentation (In French, sorry) http://qt.developpez.com/doc/4.7/qnetworkaccessmanager/
And they speak about requests sended by the server to receive replies.
But here I'm doing the opposite.
Yes, I know. You can't use QNAM. When I was suggesting that I was under impression you were interested in the client side (because of your interest in QAbstractSocket instead of QTcpServer).

erqsor
17th January 2012, 00:23
OK, you say that I'm interested by the client side because I'm using sockets.
So you think I don't need sockets in my server application.
So I'm interested to know, when a connexion is etablished with a client by the QTcpServer, how do you send data to this client ?
personaly, I don't know how to do this without using the socket of this client...

wysota
17th January 2012, 00:39
OK, you say that I'm interested by the client side because I'm using sockets.
No. I thought you were interested in the client side because QAbstractSocket represents a client side of the connection. The server is implemented by Q*Server.


So I'm interested to know, when a connexion is etablished with a client by the QTcpServer, how do you send data to this client ?
personaly, I don't know how to do this without using the socket of this client...
I'm not saying you shouldn't be using sockets! I'm saying that you should give your users an API that more resembles QNetworkReply than QTcpSocket. Internally you will use a tcp socket but you shouldn't expose that socket to the outside world.

erqsor
17th January 2012, 00:45
OK, so can you tell me how can I replace my QTcpServer by a QNetworkReply ? The QtAssistant doesn't illustrate this case...
For now, I'm using the QTcpServer::listen function for waiting client connexions, and creating a QTcpSocket to communicate with this client.

wysota
17th January 2012, 00:47
OK, so can you say to me how can I replace my QTcpServer by a QNetworkReply ? The QtAssistant doesn't illustrate this case...
For now, I'm using the QTcpServer::listen function for waiting client connexions, and creating a QTcpSocket to communicate with this client.

Come on. Read my posts carefully, I never said you should replace QTcpServer with QNetworkReply.

erqsor
17th January 2012, 01:13
Ok, misunderstanding, sorry...


No. I thought you were interested in the client side because QAbstractSocket represents a client side of the connection. The server is implemented by Q*Server.
But I need sockets in server side ^^

Anyway, I wanted to do my API like this : The user uses these two ws objects (QWsServer, QWsSocket) as he uses these two tcp objects (QTcpServer, QTcpSocket).
The user don't use tcp objects: The TCP instructions are done in background (hidden).

As I said at begining, I have some problems :
- Inherit the QWsServer from QTcpServer : I can't because the newConnection signal is already used in background by the QTcpServer. And I can't offer this signal to the QWsServer users by inheritance.
- Inherit the QWsSocket from QTcpSocket : I can't for the same reason with the readyRead signal.

So I have not used inheritance, but aggregation (I don't know if thats the right word in English). So the QWsServer has a ptr to a QTcpServer as member. And the QWsSocket has a QTcpSocket ptr as member too.
QWsServer inherits from QObject and QWsSocket inherits from QAbstratSocket.
And finaly, for now I created my own signals and fonctions to read data in a QWsSocket, but I wanted to use the QAbstractSocket features to the fullest, and so reimplementing the common read functions and signals.

I hope what I said is clear !

wysota
17th January 2012, 02:11
As I said at begining, I have some problems :
- Inherit the QWsServer from QTcpServer : I can't because the newConnection signal is already used in background by the QTcpServer. And I can't offer this signal to the QWsServer users by inheritance.
- Inherit the QWsSocket from QTcpSocket : I can't for the same reason with the readyRead signal.

So I have not used inheritance, but aggregation (I don't know if thats the right word in English). So the QWsServer has a ptr to a QTcpServer as member. And the QWsSocket has a QTcpSocket ptr as member too.
QWsServer inherits from QObject and QWsSocket inherits from QAbstratSocket.
And finaly, for now I created my own signals and fonctions to read data in a QWsSocket, but I wanted to use the QAbstractSocket features to the fullest, and so reimplementing the common read functions and signals.

That's all ok however in my opinion there is just no point in deriving your QWsSocket from QAbstractSocket. QIODevice would do just fine and you could reimplement only those methods you really needed (unfortunately not only the two you mentioned in the beginning).

And here is why I think the way I think... Unless you want to implement the client side as well, your QWsSocket won't sanely implement the following methods from QAbstractSocket:


void connectToHost ( const QString & hostName, quint16 port, OpenMode openMode = ReadWrite )
void connectToHost ( const QHostAddress & address, quint16 port, OpenMode openMode = ReadWrite )
QNetworkProxy proxy () const
void setProxy ( const QNetworkProxy & networkProxy )
bool waitForConnected ( int msecs = 30000 )
bool waitForDisconnected ( int msecs = 30000 )

So what is the point in having those methods around?

Correct me if I'm wrong, but what your users need is:

read()
write()
close()
bytesAvailable()
signal readyRead()
signal bytesWritten()
QHostAddress peerAddress()
quint16 peerPort()

Apart from the last two methods the rest is defined in QIODevice. So you are leaving 6 dangling methods just for the benefit of having other two methods that are pretty simple to implement.

A potential advantage of subclassing QAbstractSocket would be if someone wanted to replace one socket implementation for another. Unfortunately protocols differ so much that in 99% of the cases it is just not possible (and where it is possible, you can just use QIODevice API). So what is the point of deriving from QAbstractSocket?

erqsor
17th January 2012, 02:31
I developped an application with my API (So that's a project that uses my websocket server API). And I found usefull to allow server to permit connexion by using one QTcpServer and one QWsServer (So users can connect by two ways : a Qt application that uses TCP protocol, and a web application that uses websocket protocol). And next I stored the socket with a total abstraction of the protocol in a member of my client class (QAbstractSocket * socket) that can stores a QTcpSocket or a QWsSocket (never both). And I want to use the write and read methods with a total abstraction of the type of the socket to simplify and make more powerful the use of the API.
I want users uses the QWsSocket like a QTcpSocket, so I redirect the peerAddress, peerPort, state, socketType functions to the QAbstractSocket (and I need too the connected, disconnected, stateChanged signals). So if my QWsSocket doesn't inherit from QAbstractSocket, I can't use this network part of QIODevice I need.

wysota
17th January 2012, 03:14
The abstraction of the protocol probably could have as well used QIODevice instead of QAbstractSocket (since protocols care only about data exchange and not who they are talking to and how). The signal connected() is useless on the server side since when the server accepts a connection, the socket is already in the connected state. Disconnected() is pretty much useless for the server as well and even if not, adding that signal is not a problem. The only two meaningful SocketState values for the server side of the connection are Connected and Closing since the server is accepting the connection so it doesn't do host lookup, it doesn't connect and it's not listening (since it is an incoming connection). However, I'm not going to convince you anymore, it is your project and I just hope people using your code think exactly the same way you do, otherwise you'll get a lot of stupid "why doesn't it work" questions from them.

By the way, there is this thing called QLocalSocket. It's derived from QIODevice and not QAbstractSocket. To be honest I wondered many times why QUdpSocket inherits QAbstractSocket since it's neither connecting nor streaming. I wouldn't be suprised if in Qt5 QAbstractSocket simply vanished into thin air.

erqsor
17th January 2012, 11:05
Yes, you said interesting things, but I really want to construct my API like the QTcpSocket and QUdpSocket (and other sockets), That's why I did it like this. I don't want to limit my users. For a personnal project, I will do like you say, but here I think I can't do that :/

Anyway, I did some functions that works:


qint64 QWsSocket::bytesAvailable() const
{
return buffer.size();
}

qint64 QWsSocket::readData( char * data, qint64 maxSize )
{
int dataSize=buffer.size();
int i=0;
while( i<maxSize && i<dataSize )
{
data[i] = buffer.dequeue().toAscii();
i++;
}
return i;
}

qint64 QWsSocket::writeData( const char * data, qint64 maxSize )
{
return tcpSocket->write(data, maxSize);
}


QByteArray QWsSocket::readAll()
{
qint64 sz = bytesAvailable();
char * data = new char[sz];
sz = readData(data, sz);
return QByteArray(data, sz);
}

bool QWsSocket::getChar(char * c)
{
*c = 0;
if ( bytesAvailable() == 0 )
return false;

readData(c, 1);
return true;
}

qint64 QWsSocket::read(char * data, qint64 maxSize)
{
return readData(data, maxSize);
}

wysota
17th January 2012, 11:39
but I really want to construct my API like the QTcpSocket and QUdpSocket (and other sockets)
What other sockets?


I don't want to limit my users
That's what you're currently doing.



qint64 QWsSocket::bytesAvailable() const
{
return buffer.size();
}
That's incorrect. See docs for QIODevice::bytesAvailable().




QByteArray QWsSocket::readAll()
{
qint64 sz = bytesAvailable();
char * data = new char[sz];
sz = readData(data, sz);
return QByteArray(data, sz);
}
That's incorrect. readAll() is not virtual.




qint64 QWsSocket::read(char * data, qint64 maxSize)
{
return readData(data, maxSize);
}
Incorrect, read() is not virtual.

erqsor
17th January 2012, 12:38
I don't want to limit my users

That's what you're currently doing.

I'm doing like Qt.



That's incorrect. readAll() is not virtual.



Incorrect, read() is not virtual.


So what can I do ? QIODevice is useless if I cant use this functions...

wysota
17th January 2012, 13:01
I'm doing like Qt.
What "Qt does" is not always correct. And you're not "doing like Qt". Qt tends to inherit QIODevice rather than its subclasses.


So what can I do ? QIODevice is useless if I cant use this functions...
Those methods are already implemented, you don't implement them yourself. Implement readData(), writeData(), bytesAvailable(), waitForBytesWritten(), waitForReadyRead(), possibly canReadLine() and readLineData(). QAbstractSocket uses an internal "socket engine" which does most of the work. If you want to implement your own subclass of QAbstractSocket class, you have to shadow all that because you don't have a working socket engine for your protocol underneath since your QWsSocket is not a real native socket. So you have to rewrite pretty much everything QAbstractSocket does (apart from close(), isSequential() and atEnd()).

The more I look at the source code of QAbstractSocket the more I think this class was simply not meant to be subclassed outside Qt tree (not in this particular case but rather not meant to be subclassed at all). It has hardcoded support for UDP, TCP and SSL and doesn't allow hooking into the code with other implementations. QTcpSocket and QUdpSocket are simply stubs over QAbstractSocket which implements everything from both protocols itself (or actually delegates everything to a subclass of QAbstractSocketEngine such as QNativeSocketEngine).

erqsor
17th January 2012, 16:02
OK, I deleted these functions:


qint64 read( char * data, qint64 maxSize );
QByteArray readAll();
bool getChar( char * c );


But that does not work anymore.
The implementation of the others functions, can you take a look ? (only for the read part)



bool QWsSocket::atEnd() const
{
if ( buffer.size() )
return false;
return true;
}
bool QWsSocket::canReadLine() const
{
return buffer.contains('\n') || QIODevice::canReadLine();
}
bool QWsSocket::isSequential() const
{
return true;
}
qint64 QWsSocket::readData( char * data, qint64 maxSize )
{
int dataSize = buffer.size();
int i = 0;
while( i<maxSize && i<dataSize )
{
data[i] = buffer.dequeue().toAscii();
i++;
}
return i;
}
qint64 QWsSocket::readLineData( char * data, qint64 maxSize )
{
int dataSize = buffer.size();
int i = 0;
bool endLineReached = false;
while( i<maxSize && i<dataSize && !endLineReached )
{
char c = buffer.dequeue().toAscii();
if ( c == '\n' )
endLineReached = true;
data[i] = c;
i++;
}
return i;
}


Added after 9 minutes:

And I must precise that I tried to read data from the QIODevice with readAll and read(bytesAvailable) functions

wysota
17th January 2012, 16:16
qint64 QWsSocket::readData( char * data, qint64 maxSize )
{
int dataSize = buffer.size();
int i = 0;
while( i<maxSize && i<dataSize )
{
data[i] = buffer.dequeue().toAscii();
i++;
}
return i;
}
This is suboptimal (aka slow). Try this:


qint64 QWsSocket::readData( char * data, qint64 maxSize )
{
int dataSize = buffer.size(); // assuming buffer is QByteArray
int i = qMin(maxSize, dataSize);
if(i==0) return 0;
memcpy(data, buffer.data(), i);
buffer.remove(0, i);
return i;
}



qint64 QWsSocket::readLineData( char * data, qint64 maxSize )
{
int dataSize = buffer.size();
int i = 0;
bool endLineReached = false;
while( i<maxSize && i<dataSize && !endLineReached )
{
char c = buffer.dequeue().toAscii();
if ( c == '\n' )
endLineReached = true;
data[i] = c;
i++;
}
return i;
}

Suboptimal. First search for the newline and then read everything in one go.

This all doesn't change the fact that you have to override each and every method implemented by QAbstractSocket because implementations from that class are incompatible with yours.

erqsor
17th January 2012, 16:47
Thanks for the optimization.


This all doesn't change the fact that you have to override each and every method implemented by QAbstractSocket because implementations from that class are incompatible with yours.

What do you mean ?

OK, the read and readAll functions I removed they were not virtual, but what should I do instead ? I've already re-implemented all the read functions I can :/

wysota
17th January 2012, 17:11
What do you mean ?
See post #29.


OK, the read and readAll functions I removed they were not virtual, but what should I do instead ? I've already re-implemented all the read functions I can :/

Then don't limit yourself to read functions, reimplement the rest as well (you need to shadow all implementation done by QAbstractSocket, including those methods which are introduced by that class. Or... derive from QIODevice instead of QAbstractSocket :)

erqsor
17th January 2012, 17:52
Then don't limit yourself to read functions, reimplement the rest as well (you need to shadow all implementation done by QAbstractSocket, including those methods which are introduced by that class. Or... derive from QIODevice instead of QAbstractSocket :)

But you will cry if I re-implement functions that are not virtual (in QIODevice or QAbstractSocket).

wysota
17th January 2012, 23:05
I never said you were to reimplement any non-virtual methods.

erqsor
25th January 2012, 14:59
OK, I had not time to work on it for 1 week. So I will see that when I can.

Anyway, I have one change in my project, somebody asked me to enable the socket client feature. So I think I will enable it soon.

I wanted to speak about another problem. The socketDescriptor functions allow tcp server users to use the socket in another threads. My architecture don't allow this (because my QWsSocket are not real sockets, so I can't give a socketDescriptor of the WsSocket, I can't just give the socketDescriptor of the TcpSocket, but that's not a good idea (I think)). So I need to find a solution. If you have any Idea, that could be cool :)

wysota
25th January 2012, 15:14
Your socket still has a real socket descriptor, it is the TCP socket you are using. So all you need to do is to make sure that moving your socket to another thread also moves the TCP socket with it (for example by making the TCP socket a child of the websocket). So advise your users to use QObject::moveToThread() rather than manipulating the socket descriptor.

erqsor
25th January 2012, 15:27
OK, that's a good idea. Anyway I think I can't do that by another way :)
Thanks

wysota
25th January 2012, 15:33
You don't need to do anything at all, to be honest. The socket will do perfectly fine in the same thread along all the other sockets.

erqsor
25th January 2012, 16:21
OK, In my actual code, I declare the QWsSocket as the parent of the QTcpSocket, So I think I have nothing to do more. I will test that.

wysota
25th January 2012, 17:06
Be sure to be prepared that someone deletes the socket behind your back. Also bear in mind that if an object lives in thread B, you can't write to it (or read from it) from thread A. Hence if you have some kind of WsServer object, it can't access a WsSocket object directly if the latter lives in a different thread (which implies you have to implement all functionality of the active connection in the socket class and not in the server class).

erqsor
26th January 2012, 10:29
OK thanks, I will say it here if I encounter some problems.