Results 1 to 11 of 11

Thread: Client-Server application using thread pool - implementation suggestions

  1. #1
    Join Date
    Jan 2009
    Posts
    7
    Thanks
    2

    Default Client-Server application using thread pool - implementation suggestions

    Hello all!

    I am trying to implement a server application using a thread pool. I'm using QTcpSocket to handle the connections between clients and server.

    I have a very rough sketch and I would like to have your opinions, on what I could/should change to reach best solution.

    So far I have a WorkerThread that should be in charge of processing clients requests. I also have a ThreadPool class that inherits QTcpServer and reimplements incomingConnection(int socket_descriptor).

    ThreadPool has a QList of WorkerThreads that are started in the constructor (ThreadPool(QObject parent = 0 ...).


    ThreadPool
    Qt Code:
    1. const unsigned int max_worker_threads;
    2.  
    3. class ThreadPool : public QTcpServer
    4. {
    5.  
    6. public:
    7. ThreadPool(QObject *parent = 0)
    8. : QTcpServer(parent),
    9. block_size(0),
    10. active_threads(0) {
    11.  
    12. foreach(WorkerThread worker, worker_threads) {
    13. worker.start();
    14. connect(&worker, SIGNAL(thread_finished(QTcpSocket *)),
    15. this, SLOT(worker_thread_finished(QTcpSocket *)));
    16. }
    17. }
    18.  
    19.  
    20. ~ThreadPool() {
    21.  
    22. }
    23.  
    24. //signals:
    25.  
    26. slots:
    27. void pass_job() {
    28.  
    29. QTcpSocket *_socket = static_cast<QTcpSocket *>(sender());
    30.  
    31. QDataStream in(_socket);
    32. in.setVersion(QDataStream::Qt_4_4);
    33.  
    34. if (block_size == 0) {
    35. if (_socket->bytesAvailable() < (int)sizeof(quint16))
    36. return;
    37. }
    38.  
    39. in >> block_size;
    40.  
    41. if (_socket->bytes_available() < block_size) {
    42. return;
    43. }
    44.  
    45. //there is enough data in the stream
    46.  
    47. //acquire lock
    48. //mutex.lock();
    49. if (active_threads < max_threads) {
    50. active_threads++;
    51. pending_requests.append(_socket);
    52. }
    53. else {
    54. //add "job" to the list
    55. pending_requests.append(_socket);
    56. }
    57. //release lock
    58. //mutex.unlock();
    59.  
    60. //wake on worker_thread to process the request
    61. wait.wakeOne();
    62.  
    63. }
    64.  
    65. void worker_thread_finished(QTcpSocket *socket) {
    66. //mutex.lock();
    67. active_threads--;
    68. pending_requests.removeAll(socket);
    69. //mutex.unlock();
    70. //responde to the next request
    71. serve_next_request();
    72. }
    73.  
    74.  
    75.  
    76. private:
    77.  
    78. void incomingConnection(int socket_descriptor) {
    79. QTcpSocket *socket = new QTcpSocket(socket_descriptor);
    80. client_list.append(socket);
    81. connect(socket, SIGNAL(readyRead()), this, SLOT(pass_job()));
    82. }
    83.  
    84. void serve_next_request() {
    85.  
    86. if (!pending_requests.isEmpty() && active_threads < max_worker_threads) {
    87. wait.wakeOne();
    88. }
    89. }
    90.  
    91.  
    92. QList<WorkerThread *> work_threads;
    93. QList<QTcpSocket *> client_list;
    94. QList<QTcpSocket *> pending_requests;
    95.  
    96. //QMutex mutex;
    97. quint16 block_size;
    98. unsigned int active_threads;
    99. ...
    100.  
    101. };
    To copy to clipboard, switch view to plain text mode 
    WorkerThread

    Qt Code:
    1. class WorkerThread : public QThread
    2. {
    3.  
    4. public:
    5. WorkerThread(QObject parent = 0) : QThread(parent) {
    6. //do initialization
    7.  
    8. }
    9.  
    10. ~WorkerThread() {
    11.  
    12. }
    13.  
    14. void run() {
    15. while(1) {
    16. mutex.lock();
    17. wait.wait(&mutex);
    18.  
    19. //read data from socket and process request
    20.  
    21. emit thread_finished();
    22. mutex.unlock();
    23. }
    24. }
    25.  
    26. signals:
    27. void thread_finished();
    28.  
    29.  
    30. private:
    31. QMutex mutex;
    32. //other variables
    33. }
    To copy to clipboard, switch view to plain text mode 
    (I know the code is very incomplete)
    Now I have this problem: How can I pass the socket to the working_thread in order for it to process that request??

    Any comments to my "implementation" are greatly appreciated!

    Thank you very much.
    Last edited by doctore; 22nd July 2009 at 13:24. Reason: Title change

  2. #2
    Join Date
    Dec 2007
    Posts
    628
    Thanks
    3
    Thanked 89 Times in 87 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    MacOS X Unix/X11 Windows

    Default Re: Thread Pool

    Quote Originally Posted by doctore View Post
    Now I have this problem: How can I pass the socket to the working_thread in order for it to process that request??
    Have you seen the ThreadedFortuneServer qt example. I think this is very similar.

  3. #3
    Join Date
    Jan 2009
    Posts
    7
    Thanks
    2

    Default Re: Thread Pool

    Quote Originally Posted by yogeshgokul View Post
    Have you seen the ThreadedFortuneServer qt example. I think this is very similar.
    Thanks for your reply.

    But I don't think that the problems are very similar.

    The ThreadedFortuneServer, like the suggests uses threads, but in a different manner. For each new connection one new thread is created; the client request is answered and the thread terminates its execution.

    Now, what I want is something like this:

    One list of threads, that when there aren't any requests to process are sleeping (qwaitcondition). When a new request arrives one of the threads is awaken. The request is processed and the thread goes to sleep again.

  4. #4
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,359
    Thanks
    3
    Thanked 5,015 Times in 4,792 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: Client-Server application using thread pool - implementation suggestions

    Have a look at QThreadPool and QRunnable. I think they are exactly for you.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  5. #5
    Join Date
    Jan 2009
    Posts
    7
    Thanks
    2

    Default Re: Client-Server application using thread pool - implementation suggestions

    Quote Originally Posted by wysota View Post
    Have a look at QThreadPool and QRunnable. I think they are exactly for you.

    Thank you!

    I did as you suggested, and read the docs on QThreadPool and QRunnable.

    I have this sketch and a few doubts.

    Qt Code:
    1. #include <QThreadPool>
    2. #include <QRunnable>
    3.  
    4. const unsigned int num_threads = 5;
    5.  
    6.  
    7. class WServer : public QTcpServer
    8. {
    9.  
    10. public:
    11. WServer(QObject *parent = 0) : QTcpServer(parent) {
    12.  
    13. thread_pool = new QThreadPool(num_threads);
    14.  
    15. }
    16.  
    17. slots:
    18. process_request() {
    19.  
    20. socket = static_cast<QTcpSocket *>(sender());
    21. RequestProcessor *request_processor =
    22. new RequestProcessor(socket->socketDescriptor());
    23. connect(request_processor, SIGNAL(process_finished()),
    24. this, SLOT(process_finished()));
    25. thread_pool().start(request_processor);
    26.  
    27. }
    28.  
    29. process_finished() {
    30. //process_finished
    31. }
    32.  
    33. private:
    34. void incomingConnection(int socket_descriptor) {
    35. QTcpSocket *_socket = new QTcpSocket(socket_descriptor);
    36. client_list.append(_socket);
    37. connect(_socket, SIGNAL(disconnected()),
    38. this, SLOT(deleteLater()));
    39. connect(_socket, SIGNAL(readyRead()),
    40. this, SLOT(process_request()));
    41. }
    42.  
    43. QTcpSocket *socket;
    44. QThreadPool *thread_pool;
    45. QList<QTcpSocket *> client_list;
    46.  
    47. };
    48.  
    49.  
    50. class RequestProcessor : public QRunnable
    51. {
    52.  
    53. public:
    54. RequestProcessor(QObject parent = 0, int descriptor)
    55. : QRunnable(parent),
    56. socket_descriptor(descriptor) {
    57.  
    58.  
    59. }
    60. signals:
    61. void process_finished();
    62.  
    63.  
    64. private:
    65. void run() {
    66. QTcpSocket *socket = new QTcpSocket(this);
    67. socket->setSocketDescriptor(descriptor);
    68. // identify type of request
    69. // access database
    70. // send response
    71. qDebug() << "in thread: " << QThread::currentThread();
    72. emit process_finished();
    73. }
    74.  
    75. //private variables
    76. int socket_descriptor;
    77.  
    78. };
    To copy to clipboard, switch view to plain text mode 
    (I should start by saying that I'm am not in front of a computer with an installation of Qt, and the that "code" I've written is certainly not functional, :wink

    The final application should query a MySql database in response to the clients requests and send the appropriate response.

    The main doubts I have are:

    1 - Should I create one connection to the database per thread (possibly in the run() function, or the constructor of the class) or one connection is sufficient (created in the WServer class)?

    2 - Can I access members variables of the WServer class in the run function of the QRunnable class instances?? (With the access protected with QMutex). For instance, if one of the clients request the following information: what is the number of clients connected in this moment?? The run method needs to access client_list->size()

  6. #6
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,359
    Thanks
    3
    Thanked 5,015 Times in 4,792 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: Client-Server application using thread pool - implementation suggestions

    Quote Originally Posted by doctore View Post
    1 - Should I create one connection to the database per thread (possibly in the run() function, or the constructor of the class) or one connection is sufficient (created in the WServer class)?
    You need to have a dedicated connection for each thread. I'm not sure if creating them in run() if you're going to use QRunnable is a good solution. As far as I understand it a single thread can run more than one runnable. If that was the case, it would be best to store connections outside the runnable so that if a runnable is ran on a thread that used to handle some other runnable, it can pick up the connection the previous runnable created. That's purely for efficiency reasons so you can ignore it and have the connection created and destroyed within the runnable.

    2 - Can I access members variables of the WServer class in the run function of the QRunnable class instances?? (With the access protected with QMutex). For instance, if one of the clients request the following information: what is the number of clients connected in this moment?? The run method needs to access client_list->size()
    Well... that depends. If you design it properly then yes. What might be troubling is that you might cause a deadlock with improper locking. The controlling thread should be operative all the time. But if you don't intend to block it, you'll be safe.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  7. The following user says thank you to wysota for this useful post:

    doctore (22nd July 2009)

  8. #7
    Join Date
    Jan 2009
    Posts
    7
    Thanks
    2

    Default Re: Client-Server application using thread pool - implementation suggestions

    Quote Originally Posted by wysota View Post
    You need to have a dedicated connection for each thread. I'm not sure if creating them in run() if you're going to use QRunnable is a good solution. As far as I understand it a single thread can run more than one runnable. If that was the case, it would be best to store connections outside the runnable so that if a runnable is ran on a thread that used to handle some other runnable, it can pick up the connection the previous runnable created. That's purely for efficiency reasons so you can ignore it and have the connection created and destroyed within the runnable.
    Could you elaborate on this, please?

    Also, when you say that it's best to store the connections outside the runnable where precisely are you refering, and how would I inform the runabble instance which connection to use when accessing the database?

    Thank very much for the all the information.

  9. #8
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,359
    Thanks
    3
    Thanked 5,015 Times in 4,792 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: Client-Server application using thread pool - implementation suggestions

    Quote Originally Posted by doctore View Post
    Could you elaborate on this, please?

    Also, when you say that it's best to store the connections outside the runnable where precisely are you refering, and how would I inform the runabble instance which connection to use when accessing the database?
    There are two obvious possibilities:

    One is storing the connection in the runnable.
    Qt Code:
    1. class MyRunnable : public QRunnable {
    2. public:
    3.  
    4. void run(){
    5. QSqlDatabase db = QSqlDatabase::addDatabase("...", "...");
    6. doSomethingWith(db);
    7. QSqlDatabase::removeDatabase(db.connectionName());
    8. }
    9. };
    To copy to clipboard, switch view to plain text mode 

    The other is to have a map of per-thread connections
    Qt Code:
    1. QMap<Qt::HANDLE, QSqlDatabase> databases;
    2.  
    3. class MyRunnable : public QRunnable {
    4. public:
    5. void run(){
    6. Qt::HANDLE curThr = QThread::currentThreadId();
    7. if(databases.contains(curThr)){
    8. db = databases[curThr];
    9. } else {
    10. db = QSqlDatabase::addDatabase(..., ...);
    11. databases[curThr] = db;
    12. }
    13. }
    14. };
    To copy to clipboard, switch view to plain text mode 

    In this case you'd need some code to cleanup databases after threads are destroyed. QThreadStorage can probably be used for that.

    Of course the second approach only makes sense if threads are not destroyed immediately after the runnable ends but are instead reused for future runnables. That needs to be verified first.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  10. #9
    Join Date
    Jan 2009
    Posts
    7
    Thanks
    2

    Default Re: Client-Server application using thread pool - implementation suggestions

    Quote Originally Posted by wysota View Post
    There are two obvious possibilities:

    One is storing the connection in the runnable.
    Qt Code:
    1. class MyRunnable : public QRunnable {
    2. public:
    3.  
    4. void run(){
    5. QSqlDatabase db = QSqlDatabase::addDatabase("...", "...");
    6. doSomethingWith(db);
    7. QSqlDatabase::removeDatabase(db.connectionName());
    8. }
    9. };
    To copy to clipboard, switch view to plain text mode 
    The other is to have a map of per-thread connections
    Qt Code:
    1. QMap<Qt::HANDLE, QSqlDatabase> databases;
    2.  
    3. class MyRunnable : public QRunnable {
    4. public:
    5. void run(){
    6. Qt::HANDLE curThr = QThread::currentThreadId();
    7. if(databases.contains(curThr)){
    8. db = databases[curThr];
    9. } else {
    10. db = QSqlDatabase::addDatabase(..., ...);
    11. databases[curThr] = db;
    12. }
    13. }
    14. };
    To copy to clipboard, switch view to plain text mode 
    In this case you'd need some code to cleanup databases after threads are destroyed. QThreadStorage can probably be used for that.

    Of course the second approach only makes sense if threads are not destroyed immediately after the runnable ends but are instead reused for future runnables. That needs to be verified first.

    Once more thank you very much!

    One final thing, I created a small example using QThreadPool and QRunnable.
    The app is just a dialog with 4 buttons. Once any of the buttons are clicked one instance of MyRunnable class is created and passed to the start function of the ThreadPool.


    the run() function consists of a loop which prints the text of the clicked button.

    Qt Code:
    1. void MyRunnable::run() {
    2.  
    3. for(int i = 0; i < 10000; i++)
    4. qDebug() << button_text;
    5. }
    To copy to clipboard, switch view to plain text mode 
    Now, what happens if for instance, I click on the "Button A" followed by "Button B" and "BUTTON C", is that 10000 lines of "Button A" are printed followed by 10000 lines of "Button B", and 10000 lines of "BUTTON C".

    Qt Code:
    1. BUTTON A
    2. BUTTON A
    3. ...
    4. BUTTON A
    5. BUTTON B
    6. BUTTON B
    7. ...
    8. BUTTON B
    9. BUTTON C
    10. BUTTON C
    11. ...
    12. BUTTON C
    To copy to clipboard, switch view to plain text mode 
    Shouldn't the output be more like:
    Qt Code:
    1. BUTTON A
    2. BUTTON B
    3. BUTTON A
    4. BUTTON C
    5. BUTTON B
    6. .....
    To copy to clipboard, switch view to plain text mode 
    It seems that the requests are processed sequentially when they should be "simultaneously". Any ideas??

  11. #10
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,359
    Thanks
    3
    Thanked 5,015 Times in 4,792 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: Client-Server application using thread pool - implementation suggestions

    Did you set the desired number of threads in the pool? If not, then if you have a single core machine, you'll have a single thread in the pool which will result in other jobs being queued until previous ones finish.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  12. The following user says thank you to wysota for this useful post:

    doctore (22nd July 2009)

  13. #11
    Join Date
    Jan 2009
    Posts
    7
    Thanks
    2

    Default Re: Client-Server application using thread pool - implementation suggestions



    I forgot it.
    After adding the instruction setMaxThreadCount(3) everything works perfectly now.

    Thank you!

Similar Threads

  1. Thread Ownership Problem
    By tntcoda in forum Qt Programming
    Replies: 1
    Last Post: 9th June 2009, 00:18
  2. QT + OpenGL + Thread => aaaahhhhh !
    By anthibug in forum Qt Programming
    Replies: 7
    Last Post: 26th July 2008, 13:36
  3. Replies: 5
    Last Post: 17th January 2008, 21:49
  4. KDE/QWT doubt on debian sarge
    By hildebrand in forum KDE Forum
    Replies: 13
    Last Post: 25th April 2007, 06:13
  5. Problem closing a QMainWindow in Qt4.2
    By ian in forum Qt Programming
    Replies: 11
    Last Post: 17th October 2006, 00:49

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.