Results 1 to 2 of 2

Thread: Slow QTcpServer with lots of simultaneous clients

  1. #1
    Join Date
    Aug 2010
    Thanked 1 Time in 1 Post
    Qt products
    Qt4 Qt/Embedded
    Unix/X11 Windows Symbian S60 Maemo/MeeGo

    Default Slow QTcpServer with lots of simultaneous clients


    I'm writing TCP server that will serve large files. Application logic is as follows:

    1. I've subclassed QTcpServer and reimplemented incomingConnection(int)
    2. In incomingConnection, I'm creating instance of "Streamer" class
    3. "Streamer" is using QTcpSocket which is initialized with setSocketDescriptor from incomingConnection
    4. When data from client arrives, I'm sending back initial response from within readyRead() slot, and then I'm connecting socket's signal bytesWritten(qint64) to Streamer's slot bytesWritten()
    5. bytesWritten looks something like:
    Qt Code:
    1. Streamer.h:
    2. ...
    3. private:
    4. QFile *m_file;
    5. char m_readBuffer[64 * 1024];
    6. QTcpSocket *m_socket;
    7. ...
    9. Streamer.cpp
    10. ...
    11. void Streamer::bytesWritten() {
    12. if (m_socket->bytesToWrite() <= 0) {
    13. const int bytesRead = m_file->read(m_readBuffer, 64 * 1024);
    14. m_socket->write(m_readBuffer, bytesRead);
    15. }
    16. }
    17. ...
    To copy to clipboard, switch view to plain text mode 
    So basically I'm only writing new data when old data is fully written. I think that is the most asynchronous way of doing that.

    And everything works correct, except it's pretty slow when there are lots of simultaneous clients.

    With about 5 clients - I'm downloading from that server with speed around 1 MB/s (max of my home internet connection)
    With about 140 clients - download speed is around 100-200 KB/s.

    Server's internet connection is 10 Gbps and with 140 clients its use is around 100 Mbps, so I don't think that is the problem.
    Server's memory usage with 140 clients - 100 MB of 2GB available
    Server's CPU usage - max 20%
    I'm using port 800.

    When there were 140 clients on port 800 and download speed through it was like 100-200 KB/s, I've run separate copy on port 801 and was downloading at 1 MB/s without problems.

    My guess is that somehow, Qt's event dispatching (or socket notifiers?) is too slow to handle all those events.

    I've tried:
    1. Compiling whole Qt and my app with -O3
    2. Installing libglib2.0-dev and recompiling Qt (because QCoreApplication uses QEventDispatcherGlib or QEventDispatcherUNIX, so I wanted to see if there's any difference)
    3. Spawning a few threads and in incomingConnection(int) using streamer->moveToThread() depending of how much clients are currently in particular thread - that didn't make any change (though I've observed that speeds were much more varying)
    4. Spawning worker processes using
    Qt Code:
    1. main.cpp:
    2. #include <sched.h>
    4. int startWorker(void *argv) {
    5. int argc = 1;
    6. QCoreApplication a(argc, (char **)argv);
    8. Worker worker;
    9. worker.Start();
    11. return a.exec();
    12. }
    14. in main():
    15. ...
    16. long stack[16 * 1024];
    17. clone(startWorker, (char *)stack + sizeof(stack) - 64, CLONE_FILES, (void *)argv);
    To copy to clipboard, switch view to plain text mode 
    and then starting a QLocalServer in main process and passing socketDescriptors from incomingConnection(int socketDescriptor) to the worker processes. It worked correctly, but download speeds were still slow.
    5. fork()-ing process in incomingConnection() - that nearly killed the server
    6. Creating separate thread for each client - speeds dropped to 50-100 KB/s
    7. Using QThreadPool with QRunnable - no difference

    I'm using Qt 4.8.1

    I ran out of ideas.
    Is it Qt-related or maybe something with the server configuration?

    Best regards,
    Last edited by Mechan; 22nd April 2012 at 15:27.

  2. #2
    Join Date
    Aug 2010
    Thanked 1 Time in 1 Post
    Qt products
    Qt4 Qt/Embedded
    Unix/X11 Windows Symbian S60 Maemo/MeeGo

    Default Re: Slow QTcpServer with lots of simultaneous clients

    The bottleneck is definately the disk IO operations. Above 80 opened files results in server load > 1 and download speeds around 150 KB/s. Is there anything I can change in my program, or I have to play with server configuration/hardware?

Similar Threads

  1. QTcpServer with threaded clients problem
    By Witek in forum Qt Programming
    Replies: 2
    Last Post: 21st September 2011, 22:10
  2. Replies: 4
    Last Post: 30th November 2010, 21:09
  3. Extend QTcpServer to handle multiple clients
    By DiamonDogX in forum Qt Programming
    Replies: 5
    Last Post: 24th February 2010, 19:49
  4. Replies: 4
    Last Post: 23rd May 2008, 09:42
  5. selecting lots of treewidget itemns slow ...
    By jh in forum Qt Programming
    Replies: 15
    Last Post: 19th June 2006, 18:52


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.