Results 1 to 8 of 8

Thread: Many Many (most of the time sleeping) threads. Best approach?

  1. #1
    Join Date
    Apr 2015
    Posts
    8
    Thanks
    3

    Default Many Many (most of the time sleeping) threads. Best approach?

    Hi all,

    I've been looking for a similar question but I haven't found any (maybe because this is not possible).

    I am building an application that would simulate different entities executing concurrently (vehicles, people, etc). I would like each of them to have its own thread and to execute paralelly. Although being quite a big amount of elements (imagine 10000) most of the time their threads would be sleeping. I don't know if tackling this problem with QThreads is not the right way but currently I have tried different implementations:


    1. Creating all entities as QRunnable and running them all paralelly at the same time:
      I get a "Cannot create pipe main loop wake-up: too many files open" error, for running too many threads.

    2. Creating all entities as QRunnable and calling the QThreadPool to run them :
      The threadpool executes the number of threads defined by QThreadPool::globalInstance()->setMaxThreadCount() (let's say 28) and queues the rest. But even if the QRunnable attached to one of this 28 threads is sleeping, the thread wont be released and used by another queued entity (so in the end only 28 entities would be executed).

    3. I've been trying today QtConcurrent::run() but for what I've tried it seems to work in the same way QThreadPool does


    Is there any way to implement this? If QThreadPool would detach sleeping threads and run the queued awaiting ones all my problems would be fixed. Or am I approaching the problem completely wrong?

    Thanks in advance for your time.

  2. #2
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,364
    Thanks
    3
    Thanked 5,014 Times in 4,792 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: Many Many (most of the time sleeping) threads. Best approach?

    Thread pool is not a good approach as the number of threads in the pool is limited. Depending what your agents do you should either subclass QThread and implement each agent as a thread if the agent's code is synchronous or subclass QObject, implement each agent in that class and make them event-driven. Then you can either run all of them in a single thread or spawn a number of extra threads and move agent objects to different threads.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  3. The following user says thank you to wysota for this useful post:

    ander.pijoan (16th April 2015)

  4. #3
    Join Date
    Apr 2015
    Posts
    8
    Thanks
    3

    Default Re: Many Many (most of the time sleeping) threads. Best approach?

    Quote Originally Posted by wysota View Post
    Thread pool is not a good approach as the number of threads in the pool is limited. Depending what your agents do you should either subclass QThread and implement each agent as a thread if the agent's code is synchronous
    Doing so wouldn't there be the same problem as I had when running all QRunnables together?

  5. #4
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,364
    Thanks
    3
    Thanked 5,014 Times in 4,792 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: Many Many (most of the time sleeping) threads. Best approach?

    Quote Originally Posted by ander.pijoan View Post
    Doing so wouldn't there be the same problem as I had when running all QRunnables together?
    Could be. Having 1000 threads in an application is rarely a good option
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  6. #5
    Join Date
    Oct 2009
    Posts
    483
    Thanked 96 Times in 93 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Many Many (most of the time sleeping) threads. Best approach?

    If you go for the synchronous approach, make sure to adjust the stack size of each QThread to a reasonable value. Your agents probably need far less stack space than the comfortable default value.

  7. The following user says thank you to yeye_olive for this useful post:

    ander.pijoan (16th April 2015)

  8. #6
    Join Date
    Apr 2015
    Posts
    8
    Thanks
    3

    Default Re: Many Many (most of the time sleeping) threads. Best approach?

    Quote Originally Posted by wysota View Post
    Or subclass QObject, implement each agent in that class and make them event-driven. Then you can either run all of them in a single thread or spawn a number of extra threads and move agent objects to different threads.
    So for implementing this approach, you mean for example:

    - Implementing the agents just as QObject.
    - Add a battery of signal and slots for it to use by itself.
    - Once started instead of sleeping, use a timer to invoke later a signal of its own?

    Would this 10000 timers waiting for certain time to pass be a problem?

  9. #7
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,364
    Thanks
    3
    Thanked 5,014 Times in 4,792 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: Many Many (most of the time sleeping) threads. Best approach?

    Quote Originally Posted by ander.pijoan View Post
    - Once started instead of sleeping, use a timer to invoke later a signal of its own?
    Either that or rely on events (if the agent should respond upon some other agent's actions).

    Would this 10000 timers waiting for certain time to pass be a problem?
    No but I wouldn't do that. It's better that the object do "nothing". You can have one timer and connect every agent to that timer to provide a kind of "heartbeat" impulse.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  10. The following user says thank you to wysota for this useful post:

    ander.pijoan (21st April 2015)

  11. #8
    Join Date
    Apr 2015
    Posts
    8
    Thanks
    3

    Smile Re: Many Many (most of the time sleeping) threads. Best approach?

    Worked really well your proposed solution, thanks again

Similar Threads

  1. Replies: 2
    Last Post: 14th December 2011, 09:11
  2. Best approach to set a itemdelegate strategy?
    By tonnot in forum Qt Programming
    Replies: 0
    Last Post: 31st October 2011, 16:49
  3. Qt Threads vs Native Threads performance
    By StackOverflow in forum Qt Programming
    Replies: 1
    Last Post: 14th November 2010, 12:14
  4. thread - right approach?
    By heinz32 in forum Qt Programming
    Replies: 3
    Last Post: 17th June 2008, 17:39
  5. what is the best approach
    By baray98 in forum Qt Programming
    Replies: 1
    Last Post: 14th September 2007, 09:02

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.