Results 1 to 17 of 17

Thread: Producer Consumer

  1. #1
    Join Date
    Jan 2006
    Location
    Sta. Eugènia de Berga (Vic - Barcelona - Spain)
    Posts
    869
    Thanks
    70
    Thanked 59 Times in 57 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Producer Consumer

    Hi,
    I'm trying to develop an application that has 2 Threads. One Thread is a Producer and the other one is a Consumer. The two Threads have acces to two semaphores that lock them like Producer-Consumer algorithm. The Producer insert data into an array and the Consumer reads the data.

    Producer code:
    Qt Code:
    1. Producer::run()
    2. {
    3. while (bRun)
    4. {
    5. m_qUsedData->acquire();
    6. //put data into an array
    7. m_iIndex = ((m_iIndex+1)%MAX); //Jump to the next position of the array
    8. m_qFreeData->release();
    9. }
    10. }
    To copy to clipboard, switch view to plain text mode 

    Qt Code:
    1. Consumer::run()
    2. {
    3. while (bRun)
    4. {
    5. m_qFreeData->acquire();
    6. //read data from the array and do some work
    7. m_iIndex = ((m_iIndex+1)%MAX); //Jump to the next position of the array
    8. m_qUsedData->release();
    9. }
    10. }
    To copy to clipboard, switch view to plain text mode 

    When I stop the program, first I stop the Producer Thread, then the Consumer putting their "bRun" variables to false.
    The problem is that if the Consumer is locked into "m_qFreeData->acquire();" they will never exit from "run()" so it will never be really stopped.

    The program let's the user to start and stop the threads. When it stopped, other internal changes are made: the Consumer Thread reads images from a Camera and the Camera that we are acquiring images could be changed, so I have to reinitilize the Thread but the main program is stopped into ConsumerThread->wait();

    Is there anyway to unlock the Consumer Thread?
    I have also tryied to use "tryAcquire()" but it didn't work.

    Thanks,
    Òscar Llarch i Galán

  2. #2
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    5,372
    Thanks
    28
    Thanked 976 Times in 912 Posts
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Re: Producer Consumer

    Try QWaitCondition and QMutex instead of semaphores. This way you'll be able to wake up the consumer, but remember that it has to check the value of bRun after it wakes up.

  3. #3
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,359
    Thanks
    3
    Thanked 5,015 Times in 4,792 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: Producer Consumer

    If you remember to release the lock before exiting producer's run(), the consumer should be able to continue, just be sure to set some flag in the producer so that the consumer exits instead of trying to aquire the lock again. This way it should also work with semaphores.

    Of course it is easier to use a wait condition like Jacek suggested, but you'll still need to flag the consumer so that it exits before aquiring the lock again. Using wait conditions won't really change much here (I guess it would be more usefull if you had more consumers), but it'll look nicer

    In both cases you have to set the flag before the producer releases the mutex/semaphore! Otherwise the consumer might jump into the critical section once (or more times) too many.

  4. #4
    Join Date
    Jan 2006
    Location
    Sta. Eugènia de Berga (Vic - Barcelona - Spain)
    Posts
    869
    Thanks
    70
    Thanked 59 Times in 57 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Smile Re: Producer Consumer

    Hi,
    Ok, I have changed the Semaphores with two WaitConditions and a Mutex.

    Now it's working if when I stop the program don't call "wait()" of the two Threads.

    When the Producer stops (exit from "while(bRun)") I force a wakeAll to let the Consumer wake if it is waiting. When it wakes it looks the "bRun" variable and if it is true it does it's jobs and if it is false it doesn't do anything and in the "while(bRun)" it will exit. Thanks for this jacek.

    I don't understand why if I call wait() the Consumer hangs. I'm sure that the Producer calls wakeAll. Now it works and really does'nt matter this little problem, but it will be nice to know why it hangs.

    Thanks jacek, and wysota.
    Òscar Llarch i Galán

  5. #5
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    5,372
    Thanks
    28
    Thanked 976 Times in 912 Posts
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Re: Producer Consumer

    Quote Originally Posted by ^NyAw^ View Post
    I don't understand why if I call wait() the Consumer hangs.
    May we see the code?

  6. #6
    Join Date
    Jan 2006
    Location
    Sta. Eugènia de Berga (Vic - Barcelona - Spain)
    Posts
    869
    Thanks
    70
    Thanked 59 Times in 57 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Producer Consumer

    Wow !!!
    Not working!

    Producer Code:
    Qt Code:
    1. while (m_bRun)
    2. {
    3. m_pqThreadMutex->lock();
    4. if (*m_piUsedBytes == MAX_IMATGES)
    5. {
    6. m_pqBufferNotFull->wait(m_pqThreadMutex);
    7. }
    8. m_pqThreadMutex->unlock();
    9. //Put data
    10. m_iIndex = ((m_iIndex+1)%MAX);
    11.  
    12. m_pqThreadMutex->lock();
    13. (*m_piUsedBytes )++;
    14. m_pqBufferNotEmpty->wakeAll();
    15. m_pqThreadMutex->unlock();
    16. }
    17. m_pqBufferNotEmpty->wakeAll(); //Wake up the Consumer
    To copy to clipboard, switch view to plain text mode 


    Consumer Code:
    Qt Code:
    1. while (m_bRun)
    2. {
    3. m_pqThreadMutex->lock();
    4. if (*m_piUsedBytes == 0)
    5. {
    6. m_pqBufferNotEmpty->wait(m_pqThreadMutex);
    7. }
    8. m_pqThreadMutex->unlock();
    9. if (bRun) //Check if has to be stopped
    10. {
    11. //Get data
    12. m_iIndex = ((m_iIndex+1)%MAX);
    13.  
    14. m_pqThreadMutex->lock();
    15. (*m_piUsedBytes )--;
    16. m_pqBufferNotFull->wakeAll();
    17. m_pqThreadMutex->unlock();
    18. }
    19. }
    To copy to clipboard, switch view to plain text mode 

    For the Producer and Consumer Threads I call "stop()" that changes "bRun" to false. Then I call wait().

    Qt Code:
    1. void stop_Threads()
    2. {
    3. Producer->stop();
    4. Producer->wait();
    5.  
    6. Consumer->stop();
    7. Consumer->wait(); //Sometimes it hangs here
    8. }
    To copy to clipboard, switch view to plain text mode 
    Òscar Llarch i Galán

  7. #7
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    5,372
    Thanks
    28
    Thanked 976 Times in 912 Posts
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Re: Producer Consumer

    Quote Originally Posted by ^NyAw^ View Post
    Producer->stop();
    Producer->wait();
    Consumer->stop();
    Consumer->wait(); //Sometimes it hangs here
    First you tell the producer to stop, producer wakes the consumer and then you tell the consumer to stop. Now suppose that consumer is waiting to lock the mutex on line 3 and the buffer is empty. If you stop the producer, it might happen that the consumer will acquire the lock and will hang on wait condition before the main thread invokes stop(). Similar situation might happen with the producer too, so:
    1. you have to check bRun before invoking wait(),
    2. you should acquire the lock before invoking stop() to make sure that both threads are outside the critical section (as they might be inside the if statement waiting for processor time to execute wait()).

  8. #8
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    5,372
    Thanks
    28
    Thanked 976 Times in 912 Posts
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Re: Producer Consumer

    Quote Originally Posted by jacek View Post
    2. you should acquire the lock before invoking stop() to make sure that both threads are outside the critical section (as they might be inside the if statement waiting for processor time to execute wait()).
    Of course I meant the QWaitCondition::wait().

    You should invoke QThread::wait() after invoking both stop() methods.

  9. #9
    Join Date
    Jan 2006
    Location
    Sta. Eugènia de Berga (Vic - Barcelona - Spain)
    Posts
    869
    Thanks
    70
    Thanked 59 Times in 57 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Producer Consumer

    mmm,
    I don't understand you or it doesn't work.

    Could you please modify the code I posted?

    Thanks,
    Òscar Llarch i Galán

  10. #10
    Join Date
    Jan 2006
    Location
    Sta. Eugènia de Berga (Vic - Barcelona - Spain)
    Posts
    869
    Thanks
    70
    Thanked 59 Times in 57 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Producer Consumer

    Anyone know how to create this Producer-Consumer algorithm with infinite loop that let the user stop the Threads and start any times that the user want?

    I have always a Deadlock and I'm not able to see how to avoid it.

    Thanks,
    Òscar Llarch i Galán

  11. #11
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    5,372
    Thanks
    28
    Thanked 976 Times in 912 Posts
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Re: Producer Consumer

    Try this:
    Qt Code:
    1. m_pqThreadMutex->lock();
    2. Producer->stop();
    3. Consumer->stop();
    4. m_pqBufferNotFull->wakeAll();
    5. m_pqBufferNotEmpty->wakeAll();
    6. m_pqThreadMutex->unlock();
    7.  
    8. Producer->wait();
    9. Consumer->wait();
    To copy to clipboard, switch view to plain text mode 

    Qt Code:
    1. while (m_bRun)
    2. {
    3. ...
    4. if (*m_piUsedBytes == 0 && bRun ) {
    5. ...
    6. }
    7. ...
    8. }
    To copy to clipboard, switch view to plain text mode 
    Don't forget to add a test for bRun in Producer too.

  12. #12
    Join Date
    Jan 2006
    Location
    Sta. Eugènia de Berga (Vic - Barcelona - Spain)
    Posts
    869
    Thanks
    70
    Thanked 59 Times in 57 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Producer Consumer

    Ok,
    I founded where is the error but I understand why.
    In the Consumer, there is a lot of code that takes an image from the buffer and does some image processing.

    If I comment all the Processing code, the Threads work correctly but uncommenting this part of the code, the Thread hangs. The Processing code is not a Critical Section and don't have any Mutex or WaitConditions.

    Thanks for your help
    Òscar Llarch i Galán

  13. #13
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    5,372
    Thanks
    28
    Thanked 976 Times in 912 Posts
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Re: Producer Consumer

    Quote Originally Posted by ^NyAw^ View Post
    Anyone know how to create this Producer-Consumer algorithm with infinite loop that let the user stop the Threads and start any times that the user want?
    You just need another wait condition and mutex to make your threads wait for bRun to become true.

    Beware that your implementation won't work if you have more that one producer or consumer.

  14. #14
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    5,372
    Thanks
    28
    Thanked 976 Times in 912 Posts
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Re: Producer Consumer

    Quote Originally Posted by ^NyAw^ View Post
    If I comment all the Processing code, the Threads work correctly but uncommenting this part of the code, the Thread hangs.
    Do you access any shared data or GUI there?

  15. #15
    Join Date
    Jan 2006
    Location
    Sta. Eugènia de Berga (Vic - Barcelona - Spain)
    Posts
    869
    Thanks
    70
    Thanked 59 Times in 57 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Producer Consumer

    Hi,
    Do you access any shared data or GUI there?
    I access to the buffer of images to process it, then every tool (image processing tool) process the corresponding image and finally it paints to a Widget. The Widget is contains a Window (mmm... the processing library has a window object that I used to create a Widget) creating the library window into the widget and telling it that the Widget is its father.

    If I process the image but don't display it, I'm able to stop and start the program as I want to, but if I display the images into the Widget and try to stop the program it hangs.

    The code of the Widget is executed by the Consumer Thread. It Process the image, take the result image and display it.

    The Widget is in the Main Window.

    I will investigate more, and thanks for thinking on it
    Òscar Llarch i Galán

  16. #16
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    5,372
    Thanks
    28
    Thanked 976 Times in 912 Posts
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Re: Producer Consumer

    Quote Originally Posted by ^NyAw^ View Post
    The code of the Widget is executed by the Consumer Thread. It Process the image, take the result image and display it.
    And that's the problem. Only the main thread (a.k.a. the GUI thread) can touch the GUI. Better send those QImages through a queued connection to the GUI thread and let it handle them.

    Also read this carefully: http://doc.trolltech.com/4.2/threads.html
    Last edited by jacek; 17th November 2006 at 21:04. Reason: spelling error

  17. #17
    Join Date
    Jan 2006
    Location
    Sta. Eugènia de Berga (Vic - Barcelona - Spain)
    Posts
    869
    Thanks
    70
    Thanked 59 Times in 57 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Smile Re: Producer Consumer

    Hey,
    I will take a look at it.

    Thank you very much,
    Òscar Llarch i Galán

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.