How can I prevent audio application to abort?

[davidlamhauge]

I am programming a lip-sync software for animation.
I take a .wav file, and split its 44100 samples pr.sec into 25 or 30 equally big chunks, depending on chosen frames pr. sec.
I have a QTableWidget, where I can enter characters for every frame. Every character is mapped with a picture of a mouth, and when I press SPACE I hear the audio, and see the pictures in sync with the audio. No problems:)

When I want to search in the audio, frame by frame, I can do it with the up/down-arrow keys. It works if I press these keys two or three times pr. sec., but if I hold one of the keys down, the application crashes. Why?
I would like to avoid hacks like sleep-functions, but what should I do?

Here is the code that plays the active frame.

mouthFrame = activeFrame;
QFile audioFil(audioFileName);
if (audioFil.open(QIODevice::ReadOnly)){
int offset = dataOffset + 8 + ((activeFrame - 1) * samplesPerFrame * blockAlign);
audioFil.seek(offset);

ba = audioFil.read(i * samplesPerFrame * blockAlign );
audioFil.close();
QBuffer* audioBuffer = new QBuffer(&ba);
audioBuffer->open(QIODevice::ReadOnly);

output = new QAudioOutput(format,this);
output->start(audioBuffer);
output->setNotifyInterval(1000/fps);

QEventLoop loop;
connect(output, SIGNAL(stateChanged(QAudio::State)), &loop, SLOT(quit()));
connect(output, SIGNAL(notify()),this, SLOT(showMouth()));
do {
loop.exec();
} while(output->state() == QAudio::ActiveState);
output->stop();
audioBuffer->close();
delete output;
As I said, it works if you're patient...
Could it be improved?

David

[sulliwk06]

What code are you using to handle the arrow keys? Could block entry into that code until the previous arrow key has been processed?

[davidlamhauge]

I use keyPressEvent.
Here is some of the code:

void MainWindow::keyPressEvent(QKeyEvent *e){
if (!upDownRunning && framesTotal > 0){
switch (e->key()) {
case Qt::Key_Down:case Qt::Key_Right:{ // Key_Down or Key_Right = 1 frame forward
if(activeFrame < framesTotal){
upDownRunning = true;
activeFrame += 1;
showMouth(framePhonemeList[activeFrame-1]);
playwav(searchPreRoll);
}
break;
}
upDownRunning is the current "sleep"-function. It is a boolean that is set true when the key is pressed, and false when the audio is played. But it doesn't work.
searchPreRoll is a value (1 or 2) that tells how many frames to play.

[sulliwk06]

I suppose since upDownRunning isn't set to true IMMEDIATELY after its checked, it could be that the key press events are coming in so fast that the second one has passed the IF before the first has set it to true. I don't know how many instructions it takes to evaluate framesTotal > 0, switch(e->key()), and activeFrame < framesTotal. I'm probably wrong though.

[wysota]

What does playwav() do? How does it work? If it is the code you posted at the beginning then your application crashes most likely because it runs out of stack. You are spawning an event processing loop which will trigger the next key to be processed which will trigger another event processing loop, etc. You have to correct this code to use signals and slots properly.

[davidlamhauge]

What does playwav() do? How does it work? If it is the code you posted at the beginning then your application crashes most likely because it runs out of stack. You are spawning an event processing loop which will trigger the next key to be processed which will trigger another event processing loop, etc. You have to correct this code to use signals and slots properly.
Like you write, my application probably runs out of stack. Here is the header of the playWaw():

void MainWindow::playwav(int i)
{

mouthFrame = activeFrame;
QFile audioFil(audioFileName);
if (audioFil.open(QIODevice::ReadOnly)){
int offset = dataOffset + 8 + ((activeFrame - 1) * samplesPerFrame * blockAlign);
audioFil.seek(offset);



But... - since I use the keyPressEvent, and not signals and slots, how can I use signals and slots properly?

[wysota]

You need to come up with an algorithm, what your code is supposed to do, how it is supposed to behave. Currently you expect to have a linear flow in your code but this is not a good approach for an event driven framework such as Qt and even more for asynchronous operations such as sound playback. A possible approach is that you schedule sound playback, return to the event loop and when you are notified that the sound finished playing, start another note. You can queue a series of sounds and just feed them to the player when required.

[davidlamhauge]

You need to come up with an algorithm, what your code is supposed to do, how it is supposed to behave. Currently you expect to have a linear flow in your code but this is not a good approach for an event driven framework such as Qt and even more for asynchronous operations such as sound playback. A possible approach is that you schedule sound playback, return to the event loop and when you are notified that the sound finished playing, start another note. You can queue a series of sounds and just feed them to the player when required.
My problem is that my code works the way I want - except from the fact that it crashes the program...
I don't know how to "schedule" my "sound playback", but maybe it is here that I should use signals and slots?
And when you write "... start another note", is that a thread? or?
How do I "queue a series of sounds"? I don't use phonon, but maybe it is a possibility anyway...
A lot to think about.

Thanks, David

[wysota]

My problem is that my code works the way I want - except from the fact that it crashes the program...
So it doesn't work the way you want :D


I don't know how to "schedule" my "sound playback", but maybe it is here that I should use signals and slots?

The simplest way I know is to use QQueue or equivalent. The when you get signalled, check if the user currently presses the key to play the next sound and if so, take the next element from the queue.


And when you write "... start another note", is that a thread? or?
How do I "queue a series of sounds"? I don't use phonon, but maybe it is a possibility anyway...
A lot to think about.

Similar to:


struct MouthEntry {
const QString soundPath;
const QString mouthPath;
};

QQueue<MouthEntry> m_mouthQueue;

void enqueueWordToSay(const QString &word) {
for(int i=0;i<word.size();++i) m_mouthQueue = createEntryFromSound(word.at(i)); // this is oversimplified of course
}

connect(audioOutput, SIGNAL(stateChanged(...)), this, SLOT(maybePlayNextNote()));

void X::maybePlayNextNote() {
if(!iuserWantsNextFrame) { m_playing = false; }
if(m_mouthQueue.isEmpty()) { m_playing = false; }
playNote(m_mouthQueue.dequeue());
}

void X::playNote(MouthEntry entry) {
showMouth(entry.mouthPath);
play(entry.soundPath);
m_playing = true;
}

void X::keyPressEvent(QKeyEvent *ke) {
if(ke->key() == Qt::Key_Right) { userWantsNextFrame = true; maybePlayNextNote(); return; }
}

void X::keyReleaseEvent(QKeyEvent *ke) {
if(ke->key() == Qt::Key_Right) { userWantsNextFrame = false; return; }
}

etc.

[davidlamhauge]

Thank a lot!
I should have guessed that Qt had a QQueue, ready for me
I'll loook into it, when I come home from work

David