implementing network cache using qiodevice

[derektmm]

Hi,
I am trying to implement an audio HTTP stream player using the phonon framework. I can successfully play local files and even streams using a QUrl or filename as MediaSource, but this is not enough fir what i need.

The goal of my application is to play a live audio stream, that uses authentication. The authentication can be achieved by a normal http username&password, or by setting a certain value in the HTTP User Agent header. The problem is that I cannot use any of these two authentication schemes with the MediaSource url support - I cannot see any username/password support, and I cannot modify the headers of the HTTP connection.

So, I want to do the whole network part manually, connecting to the URL with a QNetworkManager, etc.. which will allow me to set the header and/or use username and password authentication.

My problem is that I think I can make this work (I have written different proof-of-concept parts of the code), but I am left with the problem that this is a live stream, therefore never ends. If I download the stream to a file, and play from there, (which is my current idea) this file will never be truncated and will end up becoming really huge after several days of play. So I need some kind of cache of the stream I'm downloading, which can delete the already played bytes and append the new downloaded bytes to the end.

My question - if this is a correct way to do what I want, which would be the best way to implement this stream cache? Using a QIODevice, as this is what I need to feed to the MediaSource.

Thanks in advance.

[high_flyer]

One way would to use a double buffer.
You fill one while you play the other, once you finished playing, you switch the buffers, and the process starts all over again.

[derektmm]

Without having tested this yet... do you know if its possible to switch the MediaSource without interrupting the playback?

As this is an endless stream, the playback will never 'finish', so I wouldn't have a moment to switch buffers. I would have to set a limit to the size of the buffer (QFIle for example), and when its reached start writing to another. As I allow the buffer to fill to a certain size before starting to play, I would have some time to write data to the new buffer, then detect when the Mediasource has consumed all data from the first buffer, then change the source with MediaSource-> setCurrentSource()

I will test this, thanks for the suggestion. If you already know if such idea can work correctly or not, please let me know.

[high_flyer]

It works well with video... should work for audio too I reckon...

[derektmm]

Hmm... when trying to implement your idea I noticed that the code was not working as expected. I'm writing the downloaded data to a file, opening with QFile and passing the qfile object as MediaSource. But it seems that it is only playing whatever is on the disk at the moment I open it - it doesn't keep reading while the data is written. I am storing a relatively large amount of data, and the bitrate of the stream is low, so I didn't notice this until now.

I realized that QNetworkReply inherits QIODevice, so I tried just passing the QNetworkReply object to the MediaSource, but it results in a segmentation fault.... :(

So I'm pretty much back to square one - do you have experience in such stream-playing code? Could you give me some pointers on how to solve this problem?

Thanks a lot for your responses.

[high_flyer]

But it seems that it is only playing whatever is on the disk at the moment I open it - it doesn't keep reading while the data is written.
I am not sure I follow.
What I meant was:
You load one QIODevice with data and start playing it.
While it is being played, you fill another QIODevice with data (which starts where the first ends).
Once the first QIODevice has come to the end, you set the second one and start playing it, and start filling the first one with data again (from the end of the second), and so on.

[derektmm]

Yes I understood what you mean. I just dont find the proper way to play from a QIODevice at the same time its being written. I have tried 2 ways to do this:

1- using a new QFile pointer to open the same file while its being written:

QFile* reader = new QFile(currentCacheFile->fileName());
source->setCurrentSource(reader);

(where currentCacheFile is the QFile* used to write the downloaded data, and source is the MediaObject)

This seems to only play the data that was written to the file when opened. All new data is not played and playback stops after a while.

2 - using the same QFile* used for writing the data. Doesnt sound like a good idea, but I tried it and it seems that the data downloaded while already playing is not written to the correct part of the file - i get scrambled sound.

source->setCurrentSource(currentCacheFile);

Am I missing the correct way to do this? Or is your idea to use small files, fille one file 100%, close it, then play it out while the other is being filled?

Thanks again for helping me out!

Added after 1 1:

Hi again,
I changed the code, using QBuffer objects instead of QFile to avoid storing on the disk.
I am now reading from the network, writing to a QBuffer, and when the QBuffer is full, i enqueue it into a QQueue of <QBuffer *>

When the MediaObject::aboutToFinish() signal is sent, I dequeue the next QBuffer, and enqueue it in the MediaObject. The slot connect to this signal is quite simple:

void StreamPlayer::aboutToFinish()
{
qDebug() << "about to finish - getting a new buffer from the queue and sending to MediaObject";
if(!bufferQueue.empty())
{
currentCacheRead = bufferQueue.dequeue();
source->enqueue(currentCacheRead);
}
}

This seems to work, but everytime there is a switch from a buffer to the next, a horrible clicking noise is heard... this is what I meant when I asked if this was possible without interrupting the playback :(

You told me you could play video in a similar way... do you know what I'm doing wrong? is it possible to adjust the code to play the sucessive buffers without any interruption?

Thanks!!

[high_flyer]

This seems to work, but everytime there is a switch from a buffer to the next, a horrible clicking noise is heard... this is what I meant when I asked if this was possible without interrupting the playback

You told me you could play video in a similar way... do you know what I'm doing wrong? is it possible to adjust the code to play the sucessive buffers without any interruption?

Regarding the clicking noise, this might have to do with phonon.
You should look in the phonon code, maybe you will have to override what happens in setCurrentSource().
Maybe phonon is resetting or reinitializing things, which you don't really need.
But I must admit my experience with Phonon is VERY limited.
I have extensive experience with high speed video, yes, however not with phonon but with engines I wrote my self.

EDIT:
One other approach would be to feed pnonon with a URL.
So you make a server locally, which loads the files and does the double buffering internally, and you give the URL to phonon.
Thus phonon only needs to be fed the source once.
In docs I see:

MediaSource::MediaSource ( QIODevice * ioDevice )

Creates a MediaSource object for the QIODevice specified by ioDevice.

This constructor can be very handy in the combination of QByteArray and QBuffer.

If you need to fetch multimedia data from a source that is not supported by MediaSource, you should subclass QIODevice and use this MediaSource constructor. It is important that you reimplement QIODevice::isSequential(), as it is used by MediaSource to determine if the media source is seekable.

ioDevice is an arbitrary readable QIODevice subclass. If the device is not opened MediaSource will open it as QIODevice::ReadOnly.
So in your case isSequential() should return false.
You might try that first as its the simplest thing to try.

Maybe someone with more Phonon experience can help you more, as maybe there are solution in phonon it self that I am no aware of for continuous streaming, but I would expect that as from what I gather from the Phonon docs, it can play such streams.

[derektmm]

Hi again,
In the end, I was about to subclass QIODevice to make my own class when I realized it would be the same as using the QNetworkReply and feeding it to the MediaSource. I had tried this before but it didn't work - this time I allowed the QNetworkReply to fill a bit more (i wait until 80kb were downloaded) and it seems to work. I get the buffering automatically, because the QNetworkReply a sequential-access qiodevice, so the read data is removed from the buffer.

So think for now this is enough for my needs. Thanks high_flyer for your interest and good ideas!