Reading problem at some (kind of random) point with QTcpSocket

**tuxiko** · 29th November 2010, 23:31

I'm not sure I understand you correctly as I always wait for "nextBlockSize". Here is the simplified server code and below I show the results of my tests. It seems a weird exception is raised.

Qt Code:

Switch view

...
    while self.socket.state() == QAbstractSocket.ConnectedState:
 
            self.nextBlockSize = 0
 
            while self.socket.bytesAvailable() < SIZEOF_UINT16:
                self.socket.waitForReadyRead(-1)
 
            print("Bytes available: %d" % self.socket.bytesAvailable())
            self.nextBlockSize = stream.readUInt16()
 
            if self.nextBlockSize == 0:
                print("GameServer: Problem with QTcpSocket?!\n"
                      "QDataStream status(): %d" % stream.status())
                return
 
            while self.socket.bytesAvailable() < self.nextBlockSize:
                self.socket.waitForReadyRead(-1)
 
            print("BlockSize: %d" % self.nextBlockSize)
...

...
    while self.socket.state() == QAbstractSocket.ConnectedState:

            self.nextBlockSize = 0

            while self.socket.bytesAvailable() < SIZEOF_UINT16:
                self.socket.waitForReadyRead(-1)

            print("Bytes available: %d" % self.socket.bytesAvailable())
            self.nextBlockSize = stream.readUInt16()

            if self.nextBlockSize == 0:
                print("GameServer: Problem with QTcpSocket?!\n"
                      "QDataStream status(): %d" % stream.status())
                return

            while self.socket.bytesAvailable() < self.nextBlockSize:
                self.socket.waitForReadyRead(-1)

            print("BlockSize: %d" % self.nextBlockSize)
...

To copy to clipboard, switch view to plain text mode

I also print at the client side what it sends (it is represented below by the byte arrays and the lines "SIZE: x" -- in red color).
The printed lines on the console are:

...
b'\x00\x14\x00\x00\x00@\x00\x00\x96\x9d\x05w\xd2\x 90\x00\x00\x03\x84\x00\x00\x01\x9f'
b'\x00\x14\x00\x00\x00@\x00\x00\x96\x9d\x05w\xd2\x 90\x00\x00\x03(\x00\x00\x01\xc7'
SIZE: 22
SIZE: 22
Bytes available: 44
BlockSize: 20
Bytes available: 22
BlockSize: 20
b'\x00\x14\x00\x00\x00@\x00\x00\x96\x9d\x05w\xd2\x 90\x00\x00\x02\xd6\x00\x00\x01\xea'
b'\x00\x14\x00\x00\x00@\x00\x00\x96\x9d\x05w\xd2\x 90\x00\x00\x02\x88\x00\x00\x02\x03'
SIZE: 22
Bytes available: 44
BlockSize: 20
Bytes available: 22
BlockSize: 20
b'\x00\x14\x00\x00\x00@\x00\x00\x96\x9d\x05w\xd2\x 90\x00\x00\x02=\x00\x00\x02\x1c'
Bytes available: 22
BlockSize: 20
SIZE: 22
b'\x00\x14\x00\x00\x00@\x00\x00\x96\x9d\x05w\xd2\x 90\x00\x00\x01\xe1\x00\x00\x02"'
Bytes available: 22
BlockSize: 20
SIZE: 22
b'\x00\x14\x00\x00\x00@\x00\x00\x96\x9d\x05w\xd2\x 90\x00\x00\x01\xc1\x00\x00\x01\xcd'
Bytes available: 22
GameServer: Problem with QTcpSocket?!
QDataStream status(): 1

(Note that the "SIZE:" lines are printed by the client GUI-thread whereas the byte arrays are printed by an independent client thread)

So at the end it seems that I have read past the end of the stream (status == 1), but my code seems correct.
However I just have caught this exception :

------------
Error in sys.excepthook:
TypeError: 'NoneType' object is not callable

Original exception was:
Traceback (most recent call last):
File "/home/tuxico/Projets/PyMagic-git/pm_gameserver.py", line 92, in run
TypeError: unorderable types: int() < NoneType()
------------

and the faulting line is in the code above:

Qt Code:

Switch view

while self.socket.bytesAvailable() < SIZEOF_UINT16:

while self.socket.bytesAvailable() < SIZEOF_UINT16:

To copy to clipboard, switch view to plain text mode

What does it means?

Thanks for all your patience.

**wysota** · 29th November 2010, 23:59

I think something might have deleted your socket.

**tbscope** · 30th November 2010, 05:54

One big problem I see is this:

Qt Code:

Switch view

self.connect(thread, SIGNAL("finished()"), thread, SLOT("deleteLater()"))

self.connect(thread, SIGNAL("finished()"), thread, SLOT("deleteLater()"))

To copy to clipboard, switch view to plain text mode

What happens if you delete or comment that line?

**tuxiko** · 30th November 2010, 21:01

By commenting this line it seems that my application does not crash easily

good catch! (I will like to have more explanations)
But the out of sync / corruption problem is not resolved (I also join another code extract below the logs) :

Note: red color is used to distinguish what the client prints

...
b'\x00\x14\x00\x00\x00@\x00\x00\xcf\x97\x04.\xb2\x d0\x00\x00\x05\xfa\x00\x00\x00\x00'
Bytes available: 22
BlockSize: 20
SIZE: 22
b'\x00\x14\x00\x00\x00@\x00\x00\xcf\x97\x04.\xb2\x d0\x00\x00\x06E\x00\x00\x00\x00'
Bytes available: 22
BlockSize: 20
SIZE: 22
b'\x00\x14\x00\x00\x00@\x00\x00\xcf\x97\x04.\xb2\x d0\x00\x00\x06q\x00\x00\x00F'
Bytes available: 22
GameServer: Bug of QTcpSocket?!
QDataStream status(): 1
...

--> the status seems to say that there has been a reading that has begun too soon :/

And another log that show a corruption case:

Notes:
- at some point, I have moved a QGraphicsItem very quickly in order to generate a lot of network packets
- the actions 2827 or 2828 are not valid numbers (valid ones are 0 and 1)
- the repetition of 2827 and 2828 seems strange...

...
b'\x00\x14\x00\x00\x00@\x00\x02f \x047:\xd0\x00\x00\x03\xa4\x00\x00\x02K'
Bytes available: 22
BlockSize: 20
SIZE: 22
b'\x00\x14\x00\x00\x00@\x00\x02f \x047:\xd0\x00\x00\x03?\x00\x00\x02D'
Bytes available: 22
BlockSize: 20
SIZE: 22
b'\x00\x14\x00\x00\x00@\x00\x02f \x047:\xd0\x00\x00\x03\x11\x00\x00\x02\x03'
Bytes available: 22
BlockSize: 20
SIZE: 22
b'\x00\x14\x00\x00\x00@\x00\x02f \x047:\xd0\x00\x00\x03!\x00\x00\x01\xa8'
Bytes available: 22
BlockSize: 20
SIZE: 22
SIZE: 22
b'\x00\x14\x00\x00\x00@\x00\x02f \x047:\xd0\x00\x00\x03c\x00\x00\x01P'
b'\x00\x14\x00\x00\x00@\x00\x02f \x047:\xd0\x00\x00\x03\xb5\x00\x00\x00\xfc'
Bytes available: 22
BlockSize: 20
Bytes available: 4118
BlockSize: 20
Bytes available: 4096
BlockSize: 20
Bytes available: 4074
BlockSize: 20
Bytes available: 4052
BlockSize: 20
Bytes available: 4030
BlockSize: 2828
GameSeverThread: action [2827] unhandled!
Bytes available: 4020
BlockSize: 2827
GameSeverThread: action [2828] unhandled!
Bytes available: 4010
BlockSize: 2828
GameSeverThread: action [2827] unhandled!
Bytes available: 4000
BlockSize: 2827
GameSeverThread: action [2828] unhandled!
Bytes available: 3990
BlockSize: 2828
GameSeverThread: action [2827] unhandled!
Bytes available: 3980
BlockSize: 2827
...

Here are the code of the client thread that sends the byte arrays:

Qt Code:

Switch view

class HandleClientRequestThread(QThread):
 
    def __init__(self, socket, request, rqLock, parent):
        super(HandleClientRequestThread, self).__init__(parent)
        self.socket = socket
        self.rqLock = rqLock
        self.request = request
        self.__toSend = []
 
    def run(self):
 
        while True:
 
            self.rqLock.lock()
 
            length = len(self.request)
 
            while length > 0:
                r = self.request.pop(0)
                print(r)
                if self.socket.state() == QAbstractSocket.ConnectedState:
                    self.socket.write(r)
                else:
                    print("socket is not connected")
 
                length = len(self.request)
 
            self.rqLock.unlock()
            self.msleep(10)

class HandleClientRequestThread(QThread):
    
    def __init__(self, socket, request, rqLock, parent):
        super(HandleClientRequestThread, self).__init__(parent)
        self.socket = socket
        self.rqLock = rqLock
        self.request = request
        self.__toSend = []

    def run(self):

        while True:

            self.rqLock.lock()

            length = len(self.request)

            while length > 0:
                r = self.request.pop(0)
                print(r)
                if self.socket.state() == QAbstractSocket.ConnectedState:
                    self.socket.write(r)
                else:
                    print("socket is not connected")

                length = len(self.request)
 
            self.rqLock.unlock()
            self.msleep(10)

To copy to clipboard, switch view to plain text mode

The client thread read the list "request" where packets are stored into by the method issueRequest() below. This later method is called
through a slot that is connected to a signal sent by a QGraphicsItem when this one has moved.

Qt Code:

Switch view

class GameClient(QWidget):
 
    def __init__(self, gameWindow):
        super(GameClient, self).__init__(None)
 
        self.socket = QTcpSocket()
        self.rqLock = QMutex()
 
         self.myPlayerId = 0
        self.resetWork()
 
        self.connect(self.socket, SIGNAL("connected()"), self.startNetworkThread)
        self.connect(self.socket, SIGNAL("readyRead()"), self.readFirstResponse)
        self.connect(self.socket, SIGNAL("disconnected()"), self.serverHasStopped)
        self.connect(self.socket, SIGNAL("error(QAbstractSocket::SocketError)"),
                     self.serverHasError)
 
 
    def startNetworkThread(self):
 
        self.thread = HandleClientRequestThread(self.socket, self.request,
                                                self.rqLock, self)
        self.connect(self.thread, SIGNAL("finished()"),
                     self.thread, SLOT("deleteLater()"))
        self.thread.start()
 
 
    def issueRequest(self, action, params):
 
        req = QByteArray()
 
        stream = QDataStream(req, QIODevice.WriteOnly)
        stream.setVersion(QDataStream.Qt_4_6)
        stream.writeUInt16(0)
        stream.writeUInt16(action)
        stream.writeUInt16(self.gameId)
        stream.writeUInt32(self.myPlayerId)
 
        for arg in params:
            arg.writeOnStream(stream)
 
        stream.device().seek(0)
 
        print("SIZE: %d" % req.size())
 
        stream.writeUInt16(req.size() - SIZEOF_UINT16)
 
        self.rqLock.lock()
        self.request.append(req)
        self.rqLock.unlock()

class GameClient(QWidget):

    def __init__(self, gameWindow):
        super(GameClient, self).__init__(None)
        
        self.socket = QTcpSocket()
        self.rqLock = QMutex()

         self.myPlayerId = 0
        self.resetWork()

        self.connect(self.socket, SIGNAL("connected()"), self.startNetworkThread)
        self.connect(self.socket, SIGNAL("readyRead()"), self.readFirstResponse)
        self.connect(self.socket, SIGNAL("disconnected()"), self.serverHasStopped)
        self.connect(self.socket, SIGNAL("error(QAbstractSocket::SocketError)"),
                     self.serverHasError)


    def startNetworkThread(self):

        self.thread = HandleClientRequestThread(self.socket, self.request,
                                                self.rqLock, self)
        self.connect(self.thread, SIGNAL("finished()"),
                     self.thread, SLOT("deleteLater()"))
        self.thread.start()


    def issueRequest(self, action, params):

        req = QByteArray()

        stream = QDataStream(req, QIODevice.WriteOnly)
        stream.setVersion(QDataStream.Qt_4_6)
        stream.writeUInt16(0)
        stream.writeUInt16(action)
        stream.writeUInt16(self.gameId)
        stream.writeUInt32(self.myPlayerId)

        for arg in params:
            arg.writeOnStream(stream)

        stream.device().seek(0)

        print("SIZE: %d" % req.size())

        stream.writeUInt16(req.size() - SIZEOF_UINT16)

        self.rqLock.lock()
        self.request.append(req)
        self.rqLock.unlock()

To copy to clipboard, switch view to plain text mode

I hope you will have some new hints to resolve my problem.

Best regards.

**wysota** · 30th November 2010, 23:08

In general something causes your socket to go out of "connected" state which causes your thread to exit which causes it to be deleted (as tbscope noticed) which causes you to access a stale object upon receiving data from the socket. First thing you should do is to get rid of the state check. It'd be best if you turned your synchronous approach into asynchronous that uses signals and slots so that you can get rid of the thread and possibly solve all your problems in one go.

**tuxiko** · 1st December 2010, 07:28

I've already modify the client to send packets without the intermediary thread, and it does not resolve the problem :/ (Moreover if my client is disconnected it will print this fact).
So there was just threads at the server side, and checking the state here is precisely good in order to avoid accessing the socket if it is disconnected.

(BTW, my client is more asynchronous with a threading approach than without one)

**wysota** · 1st December 2010, 08:47

I don't think modifying the client will solve the server's problems...

**tuxiko** · 1st December 2010, 09:14

I suppose that it is a problem at the server side, but maybe packets are "incorrectly" sent at the client side.
I will code the server in a mono-threaded approach to see if it resolves problems, but I will like to know what is wrong with my current code.

Thanks for your help.

**wysota** · 1st December 2010, 11:28

In short you are probably assuming too many things. Client and server are autonomous systems and they have to be treated like that.

Thread: Reading problem at some (kind of random) point with QTcpSocket

Thread Tools

Search Thread

Display

Hybrid View

Re: Reading problem at some (kind of random) point with QTcpSocket

Re: Reading problem at some (kind of random) point with QTcpSocket

Re: Reading problem at some (kind of random) point with QTcpSocket

Re: Reading problem at some (kind of random) point with QTcpSocket

Re: Reading problem at some (kind of random) point with QTcpSocket

Re: Reading problem at some (kind of random) point with QTcpSocket

Re: Reading problem at some (kind of random) point with QTcpSocket

Re: Reading problem at some (kind of random) point with QTcpSocket

Re: Reading problem at some (kind of random) point with QTcpSocket

Similar Threads

Howto: generate random Colors for multiple curves (random numbers)

Problem with reading in data from a QTCPSocket

QTcpSocket writing reading problem

Reading non-ASCII lines from QTcpSocket via readLine()

QTcpServer not reading as fast a QTcpSocket writing.

Tags for this Thread

Bookmarks

Bookmarks

Posting Permissions