Results 1 to 9 of 9

Thread: Qt multi-thread application freezes and several threads wait for the same mutex

  1. #1
    Join Date
    Mar 2019
    Posts
    4
    Thanks
    1
    Qt products
    Qt5
    Platforms
    Unix/X11 Windows

    Default Qt multi-thread application freezes and several threads wait for the same mutex

    I encountered a strange problem with my Qt-based multi-thread application. After several days running, the application will freeze without any response.

    After freeze occurred, I can confirm that several threads, including the main thread, are in futex_wait_queue_me status. When I attach to that application to investigate thread status by GDB, the backtrace of those threads
    shows that they all stopped at the following function with the same argument i.e. futex=0x45a2f8b8 <main_arena>.

    __lll_lock_wait_private (futex=0x45a2f8b8 <main_arena>)

    I know that on Linux, using non-asynchronous-safe functions within signal handlers is one of possible reasons for this status, i.e. several threads wait for the same mutex, (I can confirm from backtrace that they all stopped at malloc()/free() related function calls), but after I confirmed my Qt application, I can not find implementations related to Linux signal handlers. (but I am not sure whether Qt core library is using Linux signal handlers in its signal/slot mechanism.)

    I am sorry that I can not provide source code for this question because it is a huge project. Would you like tell me some possible reasons for this phenomenon, or some advises on how to debug it?

    Thanks in advance.

    I can provide backtrace, but sorry I have to delete some sensible information.

    Backtrace of sub thread:
    Qt Code:
    1. #0 in __lll_lock_wait_private (futex=0x4ad078b8 <main_arena>)
    2. #1 in __GI___libc_malloc (bytes=32) at malloc.c:2918
    3.  
    4. #11 in SystemEventImp::event(QEvent*) ()
    5. #12 in QApplicationPrivate::notify_helper(QObject*, QEvent*) ()
    6. #13 in QApplication::notify(QObject*, QEvent*) ()
    7. #14 in QCoreApplication::notifyInternal(QObject*, QEvent*) ()
    8. #15 in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) ()
    9. #16 in QCoreApplication::sendPostedEvents (receiver=0x0, event_type=0) at kernel/qcoreapplication.cpp:1329
    10. #17 in QWindowSystemInterface::sendWindowSystemEvents (flags=...) at kernel/qwindowsysteminterface.cpp:560
    11. #18 in QUnixEventDispatcherQPA::processEvents (this=0x8079958, flags=...) at eventdispatchers/qunixeventdispatcher.cpp:70
    12. #19 in QEventLoop::processEvents (this=0xbfffef50, flags=...) at kernel/qeventloop.cpp:136
    13. #20 in QEventLoop::exec (this=0xbfffef50, flags=...) at kernel/qeventloop.cpp:212
    14. #21 in QCoreApplication::exec () at kernel/qcoreapplication.cpp:1120
    15. #22 in QGuiApplication::exec () at kernel/qguiapplication.cpp:1220
    16. #23 in QApplication::exec () at kernel/qapplication.cpp:2689
    17. #24 in main(argc=2, argv=0xbffff294)
    To copy to clipboard, switch view to plain text mode 

    Backtrace of main thread:
    Qt Code:
    1. #0 in __lll_lock_wait_private (futex=0x4ad078b8 <main_arena>) at ../ports/sysdeps/unix/sysv/linux/arm/nptl/lowlevellock.c:32
    2. #1 in __GI___libc_malloc (bytes=8) at malloc.c:2918
    3. ... ...
    4. #15 in QGraphicsView::paintEvent(QPaintEvent*) ()
    5. #16 in QWidget::event(QEvent*) ()
    6. #17 in QFrame::event(QEvent*) ()
    7. #18 in QGraphicsView::viewportEvent(QEvent*) ()
    8. #19 in Platform::Drawing::GraphicsView::viewportEvent(QEvent*) ()
    9. #20 in QAbstractScrollAreaFilter::eventFilter(QObject*, QEvent*) ()
    10. #21 in QCoreApplicationPrivate::cancel_handler(QObject*, QEvent*) ()
    11. #22 in QApplicationPrivate::notify_helper(QObject*, QEvent*) ()
    12. #23 in QApplication::notify(QObject*, QEvent*) ()
    13. #24 in QCoreApplication::notifyInternal(QObject*, QEvent*) ()
    14. #25 in QWidgetPrivate::drawWidget(QPaintDevice*, QRegion const&, QPoint const&, int, QPainter*, QWidgetBackingStore*) [clone .part.175] ()
    15. #26 in QWidgetBackingStore::sync() ()
    16. #27 in QWidgetPrivate::syncBackingStore() ()
    17. #28 in QWidget::event(QEvent*) ()
    18. #29 in QApplicationPrivate::notify_helper(QObject*, QEvent*) ()
    19. #30 in QApplication::notify(QObject*, QEvent*) ()
    20. #31 in QCoreApplication::notifyInternal(QObject*, QEvent*) ()
    21. #32 in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) ()
    22. #33 in QCoreApplication::sendPostedEvents (receiver=0x809ea50, event_type=77)
    23. #34 in QGraphicsViewPrivate::dispatchPendingUpdateRequests (this=0x80e4418)
    24. #35 in QGraphicsScenePrivate::_q_processDirtyItems (this=0x80de238) at graphicsview/qgraphicsscene.cpp:508
    25. #36 in QGraphicsScene::qt_static_metacall (_o=0x80d1a80, _c=QMetaObject::InvokeMetaMethod, _id=15, _a=0x865e238)
    26. #37 in QMetaCallEvent::placeMetaCall (this=0x898d020, object=0x80d1a80)
    27. #38 in QObject::event (this=0x80d1a80, e=0x898d020) at kernel/qobject.cpp:1070
    28. #39 in QGraphicsScene::event (this=0x80d1a80, event=0x898d020) at graphicsview/qgraphicsscene.cpp:3478
    29. #40 in QApplicationPrivate::notify_helper (this=0x8077ba0, receiver=0x80d1a80, e=0x898d020) at kernel/qapplication.cpp:3457
    30. #41 in QApplication::notify (this=0x8077970, receiver=0x80d1a80, e=0x898d020) at kernel/qapplication.cpp:2878
    31. #42 in QCoreApplication::notifyInternal (this=0x8077970, receiver=0x80d1a80, event=0x898d020) at kernel/qcoreapplication.cpp:867
    32. #43 in QCoreApplication::sendEvent (receiver=0x80d1a80, event=0x898d020) at ../../include/QtCore/../../src/corelib/kernel/qcoreapplication.h:232
    33. #44 in QCoreApplicationPrivate::sendPostedEvents (receiver=0x0, event_type=0, data=0x8073318) at kernel/qcoreapplication.cpp:1471
    34. #45 in QCoreApplication::sendPostedEvents (receiver=0x0, event_type=0) at kernel/qcoreapplication.cpp:1329
    35. #46 in QWindowSystemInterface::sendWindowSystemEvents (flags=...) at kernel/qwindowsysteminterface.cpp:560
    36. #47 in QUnixEventDispatcherQPA::processEvents (this=0x8079958, flags=...) at eventdispatchers/qunixeventdispatcher.cpp:70
    37. #48 in QEventLoop::processEvents (this=0xbfffef50, flags=...) at kernel/qeventloop.cpp:136
    38. #49 in QEventLoop::exec (this=0xbfffef50, flags=...) at kernel/qeventloop.cpp:212
    39. #50 in QCoreApplication::exec () at kernel/qcoreapplication.cpp:1120
    40. #51 in QGuiApplication::exec () at kernel/qguiapplication.cpp:1220
    41. #52 in QApplication::exec () at kernel/qapplication.cpp:2689
    42. #53 in main(argc=2, argv=0xbffff294)
    To copy to clipboard, switch view to plain text mode 

  2. #2
    Join Date
    Jan 2006
    Location
    Graz, Austria
    Posts
    8,416
    Thanks
    37
    Thanked 1,544 Times in 1,494 Posts
    Qt products
    Qt3 Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Qt multi-thread application freezes and several threads wait for the same mutex

    It is strange that both thread's stack claim that the execution started in main() and that both are inside the main application event processing.

    Are those additional threads using QThread?
    If yes, are you using the method of reimplementing run(), per-thread event loop or both?

    Cheers,
    _

  3. The following user says thank you to anda_skoa for this useful post:

    gzh (2nd March 2019)

  4. #3
    Join Date
    Mar 2019
    Posts
    4
    Thanks
    1
    Qt products
    Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Qt multi-thread application freezes and several threads wait for the same mutex

    Quote Originally Posted by anda_skoa View Post
    It is strange that both thread's stack claim that the execution started in main() and that both are inside the main application event processing.

    Are those additional threads using QThread?
    If yes, are you using the method of reimplementing run(), per-thread event loop or both?

    Cheers,
    _
    Thanks.

    I create a QObject( show as SystemEventImp class in backtrace), and use movetoThread() to run it in an independent thread created by QThread.
    In SystemEventImp, I register some event handler and maintain an event queue.

    I did not reimplementing run(). Besides SystemEventImp Object, I created another Object named EventManager, which will call movetoThread()
    to move to the same QThread object SystemEventImp are running. EventManager::ProcessQueue() will be called to start a infinite loop to process
    those events maintained by SystemEventImp.

  5. #4
    Join Date
    Jan 2006
    Location
    Graz, Austria
    Posts
    8,416
    Thanks
    37
    Thanked 1,544 Times in 1,494 Posts
    Qt products
    Qt3 Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Qt multi-thread application freezes and several threads wait for the same mutex

    Ok, so kind of a hybrid approach: using moveToThread() like in the case of an event loop based thread but running your own loop like in a subclassed thread.

    How to you start your own loop?
    Do you register your event handlers in the secondary thread context?

    Cheers,
    _

  6. #5
    Join Date
    Mar 2019
    Posts
    4
    Thanks
    1
    Qt products
    Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Qt multi-thread application freezes and several threads wait for the same mutex

    Hi, anda_skoa

    > Do you register your event handlers in the secondary thread context?
    When I create SystemEventImp Object, I register my event handlers, indeed, SystemEventImp have a member variable of map type to record <event, handler> pairs.
    The registration was done before moveToThread().

    >How to you start your own loop?
    It seems that the loop started by calling EventManager::ProcessQueue() directly from main thread.
    It is a legacy source code, I am not very sure about it.
    Can this can explain your first question, i.e. "It is strange that both thread's stack claim that the execution started in main() "?

  7. #6
    Join Date
    Jan 2006
    Location
    Graz, Austria
    Posts
    8,416
    Thanks
    37
    Thanked 1,544 Times in 1,494 Posts
    Qt products
    Qt3 Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Qt multi-thread application freezes and several threads wait for the same mutex

    Quote Originally Posted by gzh View Post
    It seems that the loop started by calling EventManager::ProcessQueue() directly from main thread.
    Hmm, what does that method do?
    Its name sounds like it would do the actual event processing, i.e. the loop you mentioned earlier.
    But it doesn't make sense to move the object to another thread and then still let the main thread to the work.

    Do you call start() on the thread object?

    What kind of technique are you using to make the secondary thread call the worker objects methods?

    Quote Originally Posted by gzh View Post
    Can this can explain your first question, i.e. "It is strange that both thread's stack claim that the execution started in main() "?
    No, not yet.

    I don't quite yet understand the setup you are using.
    Very uncommon to call a method on an object that has been moved to another thread.

    Cheers,
    _

  8. #7
    Join Date
    Mar 2019
    Posts
    4
    Thanks
    1
    Qt products
    Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Qt multi-thread application freezes and several threads wait for the same mutex

    Hi, anda_skoa,

    Sorry, I have to revise what I said about event processing in my application. Indeed, It works like the followings.

    1) I did not create event loop for those threads besides main thread, all threads are using the eventloop in main thread.

    2) When SystemEventImp object created, map of <event, handler> pairs will be initialized. there are many types of event, but handler
    is only SystemEventImp object, one of its member variable is a list of handler.

    3) I create QThread instance, e.x. named subThread, Besides call SystemEventImp->moveToThread(subThread), subThread::start() will launch a
    periodic timer, that timer will create timer event to force screen redraw periodically in main thread.

    4) Besides those two threads I mentioned, there are other threads that will use call QCoreApplication:ostEvent() to append event to main event queue.
    when the event's handler is SystemEventImp object, it will be handled in the subThread I created.

    I am not familiar with QThread and its scheduling policy, I want know whether my event processing method will run into a race condition and incur that freeze as I showed on my question.

  9. #8
    Join Date
    Jan 2009
    Location
    Germany
    Posts
    387
    Thanks
    101
    Thanked 15 Times in 15 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Qt multi-thread application freezes and several threads wait for the same mutex

    Hi,

    I don't understand the details of the constellation gzh described, but the most common reason for this deadlock to occur is when a mutex is never unlocked. After several days of running, this could happen imho when a thread segfaults or throws an exception, or is coded in a way that it is possible for it to bypass the thread unlock. Wouldn't it be easier to approach this problem from a code analysis perspective? So far, no source has been shown...

  10. #9
    Join Date
    Jan 2006
    Location
    Graz, Austria
    Posts
    8,416
    Thanks
    37
    Thanked 1,544 Times in 1,494 Posts
    Qt products
    Qt3 Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Qt multi-thread application freezes and several threads wait for the same mutex

    Quote Originally Posted by gzh View Post
    3) I create QThread instance, e.x. named subThread, Besides call SystemEventImp->moveToThread(subThread), subThread::start() will launch a
    periodic timer, that timer will create timer event to force screen redraw periodically in main thread.
    How is it launching that timer and why is the subThread doing that when the timer needs to trigger work on the main thread?

    Which method of the SystemEventImp is being called by the subThread and how?

    Quote Originally Posted by gzh View Post
    I am not familiar with QThread and its scheduling policy, I want know whether my event processing method will run into a race condition and incur that freeze as I showed on my question.
    I doubt this is a matter of scheduling policy and that is handled by the operating system anyway.

    It is far more likely that you are dealing with a situation where things are not executed by the thread you think they are.

    For example you say that the main thread is executing EventManager::ProcessQueue() but you also say you have moved the object to another thread.

    What does the other thread use the object for? Which of its methods does it call?

    Cheers,
    _

Similar Threads

  1. Replies: 3
    Last Post: 6th July 2015, 19:23
  2. How to use mutex in Qt multithreaded application
    By arunkumaraymuo1 in forum Qt Programming
    Replies: 9
    Last Post: 28th February 2013, 09:13
  3. GUI freezes in loop: Thread does not help?
    By Matty23 in forum Qt Programming
    Replies: 4
    Last Post: 25th November 2010, 22:04
  4. Thread mutex lock and return value
    By ^NyAw^ in forum Qt Programming
    Replies: 5
    Last Post: 13th February 2010, 08:32
  5. How to wait for events produced by non-Qt threads?
    By Palpatine in forum Qt Programming
    Replies: 3
    Last Post: 21st May 2009, 22:45

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.