Results 1 to 11 of 11

Thread: Spliting data aquisition/visualisation into threads

  1. #1
    Join Date
    Mar 2013
    Location
    Poland
    Posts
    8
    Thanks
    2
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default Spliting data aquisition/visualisation into threads

    Hi,

    Im currently working on a program to visualise data from several sensor like accelerometers, gyroscopes, etc.
    Because Im not too familiar with QT programming and multithreading I am not too sure about best way to split my programm into threads and how to manage data synchronisation.

    My current idea is to put all data acquisition, in a fact reading from the serial port, into seperate thread, lets call it Serial. The second thread is a DataProcessor, which task is to pre-process these data, split it and send accordingly to different QWT plots and to QGLViewer(OpenGl model visualisation).
    As far as I know, using widgets(like QwtPlot) in QT is possible only in main thread. Hence the data from DataProcessor have to be send using signals-slot mechanism.

    Im afraid that doing it in this way will lead to significantly slowing down the main thread and the user interface.

    Because I dont have too much time for testing several solutions I would like to ask you about opinion of presented idea. Please let me know if I am wrong in some point or give some suggestions how can I do somtehing better.

    Thanks a lot for any replies!

  2. #2
    Join Date
    Sep 2011
    Posts
    1,245
    Thanks
    3
    Thanked 127 Times in 126 Posts
    Qt products
    Qt4
    Platforms
    Windows

    Default Re: Spliting data aquisition/visualisation into threads

    As long as you manage ownership of data correctly then you can send the data using pointers. Your situation shouldn't necessarily have any problems keeping the ui responsive.
    If you have a problem, CUT and PASTE your code. Do not retype or simplify it. Give a COMPLETE and COMPILABLE example of your problem. Otherwise we are all guessing the problem from a fabrication where relevant details are often missing.

  3. The following user says thank you to amleto for this useful post:

    robodude (30th March 2013)

  4. #3
    Join Date
    Oct 2006
    Posts
    75
    Thanks
    10
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Spliting data aquisition/visualisation into threads

    Hi,
    I am also having similar situation here with me. It would be of great help if you (amleto) put some more light on this:
    As long as you manage ownership of data correctly then you can send the data using pointers.
    . Thanks in advance.

    swamy.

  5. #4
    Join Date
    Mar 2009
    Location
    Brisbane, Australia
    Posts
    7,734
    Thanks
    13
    Thanked 1,610 Times in 1,537 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows
    Wiki edits
    17

    Default Re: Spliting data aquisition/visualisation into threads

    Reading from a serial port, splitting data, and updating widgets can all be done without threads. You do not want the extra hassle of threads if you do not need it.

    If you do use threads:
    Accessing data that is shared between threads, e.g. through a pointer to a data buffer, requires use of synchronisation mechanisms. You must ensure readers are not reading a data structure that writers may be 'simultaneously' writing to. Failing to do this is a Bad Thing that will eventually cause grief if it does not crash outright. This a general thread programming issue and not Qt specific, though Qt does provide QMutex as a suitable mechanism.

  6. The following user says thank you to ChrisW67 for this useful post:

    robodude (30th March 2013)

  7. #5
    Join Date
    Mar 2013
    Location
    Poland
    Posts
    8
    Thanks
    2
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default Re: Spliting data aquisition/visualisation into threads

    First of all, thank you guys for your opinions!

    I think I have to use threads because data from sensors will be coming "continously", measurments will be done with frequency about 500 Hz. Whats more, the data fusion will be done(for ex. Kalmann filter, etc.) In such case wouldnt it be too absorbing for one thread?

    Actually, I use such data synchronization as you mentioned, using buffer declared as a static constant size array of double and QSemaphore mechanism as described in producer-consumet in QT documentation but sharing parts of buffer instead of one element at once.

    This works well for threds Serial and DataProcessor. My problem is how to send pre-processed data form DataProcessor thread to the main thread to display it in appropriate widgets. Is the only way is using signals-slot mechanism? As far as I know I cant use mutexes/semaphores here if I want to keep GUI responsive?

    Thank you a lot!

  8. #6
    Join Date
    Sep 2011
    Posts
    1,245
    Thanks
    3
    Thanked 127 Times in 126 Posts
    Qt products
    Qt4
    Platforms
    Windows

    Default Re: Spliting data aquisition/visualisation into threads

    If there are multiple consumers of your data then signal/slots is the correct mechanism. If there is only one consumer then there is little point in using signal/slot - just use direct method calls or QMetaObject::invokeMethod to jump thread contexts. You do not need to send the actual data through a signal, though!

    Qt Code:
    1. MyDataStructure data;
    2.  
    3. // set up data
    4.  
    5. emit DataReady(data); // bad
    To copy to clipboard, switch view to plain text mode 

    Qt Code:
    1. MyDataStructure data;
    2.  
    3. // set up data
    4.  
    5. // finished writing to data - move it to some 'safe for reading' area
    6. MyDataStructure* dataPointerThatIsSafeForSharing = MoveToSharedReadingBuffer(data);
    7.  
    8. emit DataReady(dataPointerThatIsSafeForSharing); // good. signal signature is void DataReady(MyDataStructure const * ptr); so data is read only
    To copy to clipboard, switch view to plain text mode 
    you can optimise this version to only create one MyDataStructure.
    If you have a problem, CUT and PASTE your code. Do not retype or simplify it. Give a COMPLETE and COMPILABLE example of your problem. Otherwise we are all guessing the problem from a fabrication where relevant details are often missing.

  9. #7
    Join Date
    Mar 2013
    Location
    Poland
    Posts
    8
    Thanks
    2
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default Re: Spliting data aquisition/visualisation into threads

    I think I see the point.

    However could you say somthing more about this:
    Qt Code:
    1. // finished writing to data - move it to some 'safe for reading' area
    2. MyDataStructure* dataPointerThatIsSafeForSharing = MoveToSharedReadingBuffer(data);
    To copy to clipboard, switch view to plain text mode 

    Im not sure about it. Could you give some simple example what should I do in MoveToSharedReadingBuffer(data)?

  10. #8
    Join Date
    Sep 2011
    Posts
    1,245
    Thanks
    3
    Thanked 127 Times in 126 Posts
    Qt products
    Qt4
    Platforms
    Windows

    Default Re: Spliting data aquisition/visualisation into threads

    Qt Code:
    1. class ProcessAndShareData<T>
    2. {
    3. public:
    4.  
    5. void ProcessData(...);
    6.  
    7. private:
    8. T const * MoveToSharedReadingBuffer(T const & data)
    9. {
    10. T* newData = new T(data); // make a copy constructor if needed...
    11. // setup newData from data
    12. processed_data << newData;
    13.  
    14. return processed_data.back();
    15. }
    16.  
    17. private:
    18. QList<T const *> processed_data;
    19. }
    To copy to clipboard, switch view to plain text mode 

    As I say, this is a bit inefficient and you can avoid the copying of data with a bit of restructuring.
    Last edited by amleto; 30th March 2013 at 19:32.
    If you have a problem, CUT and PASTE your code. Do not retype or simplify it. Give a COMPLETE and COMPILABLE example of your problem. Otherwise we are all guessing the problem from a fabrication where relevant details are often missing.

  11. #9
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,365
    Thanks
    3
    Thanked 5,014 Times in 4,792 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: Spliting data aquisition/visualisation into threads

    I would suggest to use a circular buffer protected by two semaphores to avoid overflow and underflow. This usually works very well for quick concurrent access. Of course if you can afford to be put on hold to prevent overload (if you can't a list won't help as sooner or later you'll run out of memory).
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  12. #10
    Join Date
    Mar 2013
    Location
    Poland
    Posts
    8
    Thanks
    2
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default Re: Spliting data aquisition/visualisation into threads

    Great

    Thank you once more. Your answers were very helpful.


    Added after 7 minutes:


    Quote Originally Posted by wysota View Post
    I would suggest to use a circular buffer protected by two semaphores to avoid overflow and underflow. This usually works very well for quick concurrent access. Of course if you can afford to be put on hold to prevent overload (if you can't a list won't help as sooner or later you'll run out of memory).
    I do it in exactly same way in comunication between Serial and DataProcessor threads. I thought that it is not correct way in synchronization with the main thread.
    Last edited by robodude; 30th March 2013 at 20:24.

  13. #11
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,365
    Thanks
    3
    Thanked 5,014 Times in 4,792 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: Spliting data aquisition/visualisation into threads

    I have no idea what you do so it is not possible for me to determine if what you are doing is right or not. A circular buffer protected by two semaphores is a classic non-blocking solution for a producer-consumer problem. I'm assuming it is not what you are doing if you're asking your question (about signals and slots) here.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


Similar Threads

  1. Communication/data sharing between threads
    By Tottish in forum Newbie
    Replies: 6
    Last Post: 8th July 2013, 06:33
  2. Replies: 3
    Last Post: 17th April 2012, 09:09
  3. How do I access data between threads
    By yodasoda in forum Qt Programming
    Replies: 3
    Last Post: 26th February 2010, 19:10
  4. Sharing data between threads
    By bbui210 in forum Qt Programming
    Replies: 15
    Last Post: 19th October 2008, 17:56
  5. Sharing data across threads
    By jphn_crichton in forum Qt Programming
    Replies: 11
    Last Post: 5th May 2008, 18:29

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.