QTcpSocket leaking? What am I doing wrong?

[JohannesMunk]

Hello all!

I have been trying to pinpoint a heavy memory leak in a realtime network library I wrote, without success. As I transmit quite a lot of data (5-10MB/sec) the leak quickly crashes the application. The weird thing is that in about half of the program starts (without recompiling, just restarting!) no leaking occurs, at all! This happens both under windows (mingw/Qt4.71, msvc2008/Qt4.73) and under linux (g++/4.70).

After a lot of code rechecking and doing most of the memory allocations directly, my programm shows the following valgrind output:

==2784== 327,763,678 bytes in 398 blocks are possibly lost in loss record 284 of 284
==2784==    at 0x4025BD3: malloc (vg_replace_malloc.c:236)
==2784==    by 0x4D6E05C: qMalloc(unsigned int) (qmalloc.cpp:55)
==2784==    by 0x4D76A34: QByteArray::resize(int) (qbytearray.cpp:1379)
==2784==    by 0x4C8D5B9: QRingBuffer::reserve(int) (qringbuffer_p.h:187)
==2784==    by 0x4C8793C: QAbstractSocketPrivate::readFromSocket() (qabstractsocket.cpp:1158)
==2784==    by 0x4C87F4B: QAbstractSocketPrivate::canReadNotification() (qabstractsocket.cpp:614)
==2784==    by 0x4C74A7A: QAbstractSocketEngine::readNotification() (qabstractsocketengine.cpp:154)
==2784==    by 0x4C75CEE: QReadNotifier::event(QEvent*) (qnativesocketengine.cpp:1103)
==2784==    by 0x41B7FFB: QApplicationPrivate::notify_helper(QObject*, QEvent*) (qapplication.cpp:4396)
==2784==    by 0x41BED92: QApplication::notify(QObject*, QEvent*) (qapplication.cpp:3798)
==2784==    by 0x4E8D68A: QCoreApplication::notifyInternal(QObject*, QEvent*) (qcoreapplication.cpp:732)
==2784==    by 0x4EBDA12: socketNotifierSourceDispatch(_GSource*, int (*)(void*), void*) (qcoreapplication.h:215)

==2784== 327,763,678 bytes in 398 blocks are possibly lost in loss record 284 of 284
==2784==    at 0x4025BD3: malloc (vg_replace_malloc.c:236)
==2784==    by 0x4D6E05C: qMalloc(unsigned int) (qmalloc.cpp:55)
==2784==    by 0x4D76A34: QByteArray::resize(int) (qbytearray.cpp:1379)
==2784==    by 0x4C8D5B9: QRingBuffer::reserve(int) (qringbuffer_p.h:187)
==2784==    by 0x4C8793C: QAbstractSocketPrivate::readFromSocket() (qabstractsocket.cpp:1158)
==2784==    by 0x4C87F4B: QAbstractSocketPrivate::canReadNotification() (qabstractsocket.cpp:614)
==2784==    by 0x4C74A7A: QAbstractSocketEngine::readNotification() (qabstractsocketengine.cpp:154)
==2784==    by 0x4C75CEE: QReadNotifier::event(QEvent*) (qnativesocketengine.cpp:1103)
==2784==    by 0x41B7FFB: QApplicationPrivate::notify_helper(QObject*, QEvent*) (qapplication.cpp:4396)
==2784==    by 0x41BED92: QApplication::notify(QObject*, QEvent*) (qapplication.cpp:3798)
==2784==    by 0x4E8D68A: QCoreApplication::notifyInternal(QObject*, QEvent*) (qcoreapplication.cpp:732)
==2784==    by 0x4EBDA12: socketNotifierSourceDispatch(_GSource*, int (*)(void*), void*) (qcoreapplication.h:215)

To copy to clipboard, switch view to plain text mode 

I'm using a standard QTcpSocket.

void QntsBlockStreamReader::setupDataTCP()
{
    nextBlockSize = 0;
    dataPortTCP = new QTcpSocket();
    connect(dataPortTCP,SIGNAL(connected()),this,SLOT(dataPortConnectedTCP()));
    connect(dataPortTCP,SIGNAL(readyRead()),this,SLOT(dataIncomingTCP()));
    dataPortTCP->connectToHost(writerAddress,dataPortNo);
}

void QntsBlockStreamReader::setupDataTCP()
{
    nextBlockSize = 0;
    dataPortTCP = new QTcpSocket();
    connect(dataPortTCP,SIGNAL(connected()),this,SLOT(dataPortConnectedTCP()));
    connect(dataPortTCP,SIGNAL(readyRead()),this,SLOT(dataIncomingTCP()));
    dataPortTCP->connectToHost(writerAddress,dataPortNo);
}

To copy to clipboard, switch view to plain text mode 

In it's readyRead slot, I create block objects - which I checked are properly released after usage.

void QntsBlockStreamReader::dataIncomingTCP()
{
    while (dataPortTCP->bytesAvailable() > 0)
    {
        if (nextBlockSize == 0) {
            if (dataPortTCP->bytesAvailable() < (sizeof(quint8)+sizeof(TBlockSize))) return;
            // BlockSize contains the blockSize value itself
            QByteArray b = dataPortTCP->peek(sizeof(quint8)+sizeof(TBlockSize));
            QntsDataStream ds(&b,QIODevice::ReadOnly);
            quint8 protocolVersion;
            ds >> protocolVersion >> nextBlockSize;
        }
        if ((TBlockSize)dataPortTCP->bytesAvailable() < nextBlockSize) return;
        logMsg(QString("Bytes Available: %1 - %2").arg(dataPortTCP->bytesAvailable()).arg(QntsBlock::allocatedBlocks));
        QByteArray newData = dataPortTCP->read(nextBlockSize);
        blockArrived(new QntsBlock(newData),dataPortTCP->peerAddress(),dataPortTCP->peerPort());
        nextBlockSize = 0;
    }
}
 
QntsBlock::QntsBlock(const QByteArray& ba)
{
    m_size = ba.size();
    m_data = (char*)malloc(m_size);
    memcpy(&m_data[0],(void*)ba.constData(),m_size);
    allocatedBlocks += 1;
}
 
QntsBlock::~QntsBlock()
{
    allocatedBlocks -= 1;
    free(m_data);
}

void QntsBlockStreamReader::dataIncomingTCP()
{
    while (dataPortTCP->bytesAvailable() > 0)
    {
        if (nextBlockSize == 0) {
            if (dataPortTCP->bytesAvailable() < (sizeof(quint8)+sizeof(TBlockSize))) return;
            // BlockSize contains the blockSize value itself
            QByteArray b = dataPortTCP->peek(sizeof(quint8)+sizeof(TBlockSize));
            QntsDataStream ds(&b,QIODevice::ReadOnly);
            quint8 protocolVersion;
            ds >> protocolVersion >> nextBlockSize;
        }
        if ((TBlockSize)dataPortTCP->bytesAvailable() < nextBlockSize) return;
        logMsg(QString("Bytes Available: %1 - %2").arg(dataPortTCP->bytesAvailable()).arg(QntsBlock::allocatedBlocks));
        QByteArray newData = dataPortTCP->read(nextBlockSize);
        blockArrived(new QntsBlock(newData),dataPortTCP->peerAddress(),dataPortTCP->peerPort());
        nextBlockSize = 0;
    }
}

QntsBlock::QntsBlock(const QByteArray& ba)
{
    m_size = ba.size();
    m_data = (char*)malloc(m_size);
    memcpy(&m_data[0],(void*)ba.constData(),m_size);
    allocatedBlocks += 1;
}

QntsBlock::~QntsBlock()
{
    allocatedBlocks -= 1;
    free(m_data);
}

To copy to clipboard, switch view to plain text mode 

I would greatly appreciate any clues or hints how to proceed! First of all I would like to make sure, that I am not doing something plainly stupid, before investing even more time boiling this down, by creating a small test app, that shows the same symptoms.

Thx a lot in advance!

Johannes

[moviemax]

I didn`t get your code exactly but once I had the same problem with
mismatch the blocking functions with the non-blocking signals & slots.

If you create some Sockets within a exec() event loop you should use the
non blocking way.

read carefully the QAbstractSocket
int http://doc.qt.nokia.com/4.1/qabstrac...ected-function

[JohannesMunk]

Thank you for your time.. but, as you can see in the code I provided, I am using the readyRead-signal and direct reading of the blockbuffer, without any blocking calls to the socket.

Furthermore, I don't think that swaping to the blocking functions will change the memory allocation behaviour of the socket, which seems to leak.

The code valgrind points out is purely Qt. Am I inadvertently holding a reference to an implicitly shared byte array? I can't see it!

Thx a lot in advance for any further hints!

Johannes

[moviemax]

So does QntsBlockStreamReader lives in a thread inside a event loop?

I this Class created in the GUI thread do you have some threads like this?

myThread::start()  
   while ( m_running) 
  {
 
 }

myThread::start()  
   while ( m_running) 
  {

 }

To copy to clipboard, switch view to plain text mode 

[JohannesMunk]

If that would be the case, then my socket could not emit its readyRead signal. But it does so beautifully.

Everything works. But sporadically leaks huge amounts of memory!

The reader and thus the tcpsocket lives in a thread like this:

QntsBlockStreamReader::QntsBlockStreamReader(QString pName,QThread* pThread)
{
    thread = pThread;
    if (thread == 0)
    {
        thread = new QThread(this);
        thread->start();
    }
 
    this->moveToThread(thread);
 
    QMetaObject::invokeMethod(this,"setupDataTCP");
}
 
void QntsBlockStreamReader::setupDataTCP()
{
    nextBlockSize = 0;
    dataPortTCP = new QTcpSocket();
    connect(dataPortTCP,SIGNAL(connected()),this,SLOT(dataPortConnectedTCP()));
    connect(dataPortTCP,SIGNAL(readyRead()),this,SLOT(dataIncomingTCP()));
    dataPortTCP->connectToHost(writerAddress,dataPortNo);
}

QntsBlockStreamReader::QntsBlockStreamReader(QString pName,QThread* pThread)
{
    thread = pThread;
    if (thread == 0)
    {
        thread = new QThread(this);
        thread->start();
    }

    this->moveToThread(thread);

    QMetaObject::invokeMethod(this,"setupDataTCP");
}

void QntsBlockStreamReader::setupDataTCP()
{
    nextBlockSize = 0;
    dataPortTCP = new QTcpSocket();
    connect(dataPortTCP,SIGNAL(connected()),this,SLOT(dataPortConnectedTCP()));
    connect(dataPortTCP,SIGNAL(readyRead()),this,SLOT(dataIncomingTCP()));
    dataPortTCP->connectToHost(writerAddress,dataPortNo);
}

To copy to clipboard, switch view to plain text mode 

I don't think that we are close to a solution here, because this would be a fundamental flaw and not a sporadic leak.

Joh

[wysota]

Please provide a minimal compilable example reproducing the problem.

By the way, creating a thread with 'this' as a parent and moving 'this' object to the thread effectively moves the thread object to itself which is wrong.

You might be leaking memory because of an incorrect use of threads and sockets. Some cleanup routine might not be firing if some piece of code gets confused by incorrect thread affinity.

[JohannesMunk]

Wysota, thanks for your input! I quickly tested your hypothesis. Inside the MainForm constructor, thus from the mainthread:

reader = new QntsBlockStreamReader("COVE BXPD",QThread::currentThread());

reader = new QntsBlockStreamReader("COVE BXPD",QThread::currentThread());

To copy to clipboard, switch view to plain text mode 

... and it still leaks! I followed your discussions concerning sockets and threading and I agree that in most cases threading a socket is plainly unnecessary. In this case I have the additional special requirement of minimal latency (~1ms) for my working code, and I can't take the chance of mainthread-stuff interfering. Furthermore this library is also intended to be used from a non qt-app, and I want the socket to handle its data even if the calling thread messes up.

I am inclined to implement the socket handling myself, without any unnecessary intermediate ringbuffering. On the other hand I am still convinced that QTCPSocket is error free and its just my usage of it that screws it up! I will try to investigate how usually its ringbuffer is released. If that does not reveal any new clues, I will try to reproduce the problem in a small test application!

Johannes

[wysota]

Try getting rid of threads completely and check if the leak is still there. And explain why you think there is a leak in the first place. Valgrind reports your memory as reachable so there is a pointer somewhere that points to those memory blocks. Maybe you're just not freeing memory yourself somewhere? That's why I asked for a minimal example reproducing the problem. BTW. I don't see how putting each socket in a separate thread helps achieving <1ms latency.

[JohannesMunk]

I did test it entirely without threads! It must be the internal ringbuffer that has some dangling pointers left. I don't know what is going on.

But I've had enough and implemented a native windows socket implementation, and instant reward: no leaks anymore!

I will hunt down this usage pattern incompatibility / bug, but not this week. The project comes first!

I will be back with this!

Johannes