View Full Version : Phonon - custom sink/source/effects

18th August 2011, 02:57
First off... Phonon is really cool!

As far as I can tell, there is no way to create a custom sink/source/effect for the Phonon pipeline. Some people, (myself included), have tried inheriting QIODevice or MediaObject, but it doesn't seem to be possible. Maybe I'm going about it the wrong way, but nothing I tried worked.

The answer most often received when someone asks about custom graph nodes is that they have to create their own Directshow/Gstreamer/VLC/Quicktime plugin. One major problem with this approach is that if you want your node to be cross platform across Mac/Win/Lin you end up creating 3 different plugins - one for Gstreamer, Quicktime, and DirectShow.

I've never tried creating a filter for Quicktime, but I have done it for DirectShow and looked at examples for Gstreamer - and it's ugly! The process for writing plugins/filters for the three systems are completely different, so you're looking at three rather steep learning curves. The process is so complicated and lacking in documentation that most people end up tweaking examples to make their custom filter instead of writing one from scratch. If you're a Linux guy, you get to muck with a mess of Microsoft COM code. If you're a Windows guy you get to play with C structs that are pretending to be C++ classes and inherit using a really funky GObject system.

To use DirectShow, you get to install a giant executable for the SDK. First it was part of DirectX, then they moved it to it's own extras package, then it was part of some server service pack, then it moved to the Windows SDK, and who knows what after that. If you want to build gstreamer you get to round up a whole bunch of libraries from all over and configure/build each one. And then... when you're all done, you have to tweak your application installer to perform special steps like attaching dependencies or registering activex objects - all depending on which platform it's running on.

So... my point of all this is: if QT had it's OWN multimedia pipeline framework to fall back on, it would make writing custom source/sink/effects so much easier! I'm not saying ditch the current backends - there are too many valuable graph nodes already written to start over, but I am suggesting that an additional QT backend for us coders to add modules could be VERY useful.

Before you call sanity police... Yes, a new multimedia pipeline system is a big project. There are only around five major ones currently in use, (GStreamer, DirectShow, Media Foundation, QuickTime, and VLC). I don't know about Quicktime or MF, but I can definitely say that it's not easy writing filters/plugins for the other pipeline systems and some of those systems are old and in need of an update.

People love Qt because it's clean object oriented C++ with great documentation. Imagine a multimedia pipeline with that philosophy. Instead of forcing people to clumsily "inherit" C structures and register COM objects, they'll inherit from a clean Qt abstract class and implement the required functions (such as for negotiating image formats or determining output pins). If they get stuck, Qt's great documentation will be right there to help.

The benefits to everyone would be immediate. If there's an easy way for people to create nodes in a multimedia pipeline system, it wouldn't take long before there were thousands of new nodes. More codecs, more filters, more displays, for file formats, and a great reason to use Qt in an audio/video application.

And now for a few notes on the implementation. I can think of two possible methods on how this might work. I'm sure the Qt and KDE guys who are a lot smarter than I can come up with other ways. Here were my ideas...

1) Set it up so that this new "Qt backed" that exists concurrently with the normal dshow/gstreamer/quicktime backend. You could still get all the benefits of the backend, in addition to the QT backend.

2) Make one single Qt "backend", and then wrap up other backends within the Qt one. The Qt "backend" would have abstract classes for source/sink/filter/pin/path that could be inherited from. Since the classes would be abstract, it would be possible to make subclasses like GStreamerSource, DShowSink, and DShowTransformFilter, etc. The DShowSourceFilter could encapsulate any one of the possible directshow sources. Sorta like how ffdshow encapsulates all of ffmpeg's formats/codecs into a few different directshow filters.

Well anyway... I hope someone at Qt or KDE finds this and can help with either a better method or with implementing it. I'm willing to help work on it but I have a separate job so I can't work on it full time.