application freezes without crashing (and closing)

[nass]

Hello everyone,
for sometime now i've been trying to debug a small application that is running on a tweaked debian distribution on an ARM processor.
I've been developing the application on a slackware 10.2 machine, then using scratchbox to compile the sources for ARM, and finally copying the binary to the target machine.

eventually the scratchbox environment was moved to another computer and installed there by a different person... so it's possible that not exactly the same toolchain versions have been installed...
note also that the libqui, libqt-mt and libqt libraries have been transferred to the target board from the first scratchbox installation.

so my problem is that on occasions - seemingly random - the application freezes, but it does not seg fault or crash for any reason.. 'ps' shows that the application is still running but it looks as if it does not receive mouse events... (the only form of input is mouse events through a touch screen)

im guess it may have had smth to do with libraries mismatch, so i copied to the target board the libs libqt-mt and libqui (which are used by the executable, something that i realised using the ldd command)... it still seems to do that... and i can't even get a core dump in case that would help!!!!

thank you in advance for your help
nass

[wysota]

Can you attach to the running proccess using gdb and checking the frame stack?

[nass]

i shall try gdbserver and doing exactly that in the following freeze..
ill let you know of the output.

[nass]

ok lets see

(gdb) frame 0
#0 0x40b82f9c in __pthread_sigsuspend () from /lib/libpthread.so.0
(gdb) l
180 main.cpp: No such file or directory.
in main.cpp
(gdb) frame 1
#1 0x40b82d7c in __pthread_wait_for_restart_signal () from /lib/libpthread.so.0
(gdb) frame 2
#2 0x40b8443c in __pthread_alt_lock () from /lib/libpthread.so.0
(gdb) frame 3
#3 0x40b81748 in pthread_mutex_lock () from /lib/libpthread.so.0
(gdb) frame 4
#4 0x409db840 in pthread_mutex_lock () from /lib/libc.so.6
(gdb) frame 5
#5 0x40501624 in QRecursiveMutexPrivate::lock () from /tmp/libs/libqt-mt.so.3
(gdb) frame 6
#6 0x40501a00 in QMutex::lock () from /tmp/libs/libqt-mt.so.3
(gdb) frame 7
#7 0x40260158 in QApplication::postEvent () from /tmp/libs/libqt-mt.so.3
(gdb) frame 8
#8 0x0000f90c in SIGHUPHandler (sig=-4) at main.cpp:59
59 in main.cpp
(gdb) frame 9
#9 0x40b85bdc in __pthread_sighandler () from /lib/libpthread.so.0
(gdb)


so i am showing some frames over here... not sure exactly what they mean though.. because i didn't really play around with mutex locks....
to me it starts making sense in frame 7 basically...

but a bit of introduction first: this application is a GUI running independent of an algorithm written in c++.. the algorithm does some processing and then sends a signal to the GUI (SIGUSR1) to notify of the updated values... the data is exchanged between the 2 processes through the use of a shared memory segment.

when the signal is intercepted by the GUI , an event is posted internally like this:


void SIGHUPHandler( int sig )
{
QApplication::postEvent( qApp, new QCustomEvent( SIGHUPEVENT ) );
}

so it waits in queue till the processor serves the event..
to carry out this functionality and with the limited events concept i was familiar with at the time, i inherited main object class, to emit a signal, the 'qt' way when the event would be served.


class MainApplication : public QApplication //inhirited the QApplication to add event post capability signalling
{
Q_OBJECT

public:
MainApplication( int &argc, char **argv ) : QApplication(argc,argv) {}

signals:
void signalUp(); //the signal definition of the application

protected:
void customEvent( QCustomEvent *ev ){
if ( ev->type() == SIGHUPEVENT )
emit signalUp();
}

events are posted by the controller approximately twice per second...
so what i'm basically 'reading' from the above gdb output, is that the SIGHUPHandler function posts an event.. and then some internal qt stuff are happening that brings us in frame 0, ie '__pthread_sigsuspend' ....

if i press 'c' for continue in gdb now...


gdb) c
Continuing.
[New Thread 16384]

Program received signal SIGSTOP, Stopped (signal).
[Switching to Thread 16384]
0x40b82f9c in __pthread_sigsuspend () from /lib/libpthread.so.0
(gdb)

thank you for your help
i hope this can help abit..
nass

[wysota]

Change the sighup handler. The signal may be intercepted while the event loop is not in a coherent state. Instead I'd suggest setting a flag which you should periodically check in your application (for example using a timer).

[nass]

hmmm yes that would be polling really the shared memory for changed data...thing is signaling is nice... takes up a little less resources... i would insist on keeping working with it...

also the event queue is a valuable asset since the update of data on screen is done 'when the cpu has available resources to spare'... (ie as a separate thread) so it adds 'points' to flexible programming..

so primarily is it possible to ensure that the event loop is in a coherent state, and if thats not possible can i get you insight on perhaps finding a way that involves the signal handler?

thank you
nass

[wysota]

hmmm yes that would be polling really the shared memory for changed data...thing is signaling is nice... takes up a little less resources... i would insist on keeping working with it...
I'm not saying you shouldn't use signals. I just say you shouldn't post an event from within a signal handler. Rising a flag is really a regular way of doing things like that... Please consider the fact that postEvent() is not a single machine instruction, thus you're not really gaining anything here. A simple

if(flagRaised){ ... }
now and then while doing something else won't harm.


also the event queue is a valuable asset since the update of data on screen is done 'when the cpu has available resources to spare'... (ie as a separate thread) so it adds 'points' to flexible programming..
Hmm... you mean you're redrawing from a non-gui thread? Are you redrawing Qt widgets like that? If so, you shouldn't - that may be the reason of the inconsistency. You should only draw from within the main (QApplication::exec) thread.


so primarily is it possible to ensure that the event loop is in a coherent state, and if thats not possible can i get you insight on perhaps finding a way that involves the signal handler?
You can always spawn a separate thread that will do nothing, just wait for signals (just make sure the signal is delivered to that particular thread), but I'd call it a waste of resources. I'd really go for rising a flag in the signal handler. Then you'll be able to post an event is some more predictable state.

[nass]

yes raising a flag sounds fair enough, ill get right down to it..
however, let me investigate a bit more what you said about the updating of the values. in the labels...

im doing someLabel->setText()

basically on intercepting the signal, i post the event, then when it gets served, it emits an internal QT signal, that is connected to a procedure that reads the data from the shared memory to a local . on end, it calls the updating function that uses setText() calls to update values on the GUI..
you think there is a problem? let me remind you that while development was carried out in the initial scratchbox installation, i had never had a freezing or crashing problem...

[wysota]

you think there is a problem? let me remind you that while development was carried out in the initial scratchbox installation, i had never had a freezing or crashing problem...

If you call setText() from the gui thread, then everything is fine. A problem might only arise if you were doing gui manipulation from within a worker thread.

[nass]

i am sorry i sound so fresher in the threads programming...
i assume by worker thread you mean the signal handler (ie pre- event queue) and gui thread the internal qt signal emission (post event queue)....

ie i see the post event as a 'buffer' between unplanned incoming signals.. that can arrive at any time... and well planned signal serving, at times when the gui can do so....

[nass]

so i did, the application is running for a day now... thats a good sign, but its not enough 'mileage' yet... past versions would manage to survive that long without freezing..
ill let you know
thank you once again for your help

[wysota]

i assume by worker thread you mean the signal handler (ie pre- event queue) and gui thread the internal qt signal emission (post event queue)....
No. Threads have nothing to do with signals. It's just you said you were doing something "in a separate thread", so I assumed you were using multiple threads.

[nass]

no i haven't implemented any threads of my own... i was under the impression that among other Qt operations that spawh separate threads, the signalling mechanism would do so too.

[wysota]

Hmm... This may come as a surprise for you, but Qt doesn't spawn threads on its own for any of the tasks it does.

[nass]

ok this goes off topic, but what is the point of libqt-mt then?

[wysota]

That applications built over Qt (like KDE) can spawn their own threads. Qt3 itself can be built without thread support and still have the same functionality (apart from ability to spawn threads of course).