Linux has historically lagged behind other operating systems in the multimedia arena. Microsoft's Windows™ and Apple's MacOS™ both have strong support for multimedia devices, multimedia content creation, playback, and realtime processing. Linux, on the other hand, has a poorly integrated collection of multimedia utilities and applications available, which can hardly compete with the professional level of software available for MS Windows and MacOS.
GStreamer was designed to provide a solution to the current Linux media problems.
We describe the typical problems in today's media handling on Linux.
The Linux user who wishes to hear a sound file must hunt through their collection of sound file players in order to play the tens of sound file formats in wide use today. Most of these players basically reimplement the same code over and over again.
The Linux developer who wishes to embed a video clip in their application must use crude hacks to run an external video player. There is no library available that a developer can use to create a custom media player.
Your typical MPEG player was designed to play MPEG video and audio. Most of these players have implemented a complete infrastructure focused on achieving their only goal: playback. No provisions were made to add filters or special effects to the video or audio data.
If you want to convert an MPEG-2 video stream into an AVI file, your best option would be to take all of the MPEG-2 decoding algorithms out of the player and duplicate them into your own AVI encoder. These algorithms cannot easily be shared across applications.
Attempts have been made to create libraries for handling various media types. Because they focus on a very specific media type (avifile, libmpeg2, ...), significant work is needed to integrate them due to a lack of a common API. GStreamer allows you to wrap these libraries with a common API, which significantly simplifies integration and reuse.
Your typical media player might have a plugin for different media types. Two media players will typically implement their own plugin mechanism so that the codecs cannot be easily exchanged. The plugin system of the typical media player is also very tailored to the specific needs of the application.
The lack of a unified plugin mechanism also seriously hinders the creation of binary only codecs. No company is willing to port their code to all the different plugin mechanisms.
While GStreamer also uses it own plugin system it offers a very rich framework for the plugin developer and ensures the plugin can be used in a wide range of applications, transparently interacting with other plugins. The framework that GStreamer provides for the plugins is flexible enough to host even the most demanding plugins.
Because of the problems mentioned above, application authors have so far often been urged to spend a considerable amount of time in writing their own backends, plugin mechanisms and so on. The result has often been, unfortunately, that both the backend as well as the user interface were only half-finished. Demotivated, the application authors would start rewriting the whole thing and complete the circle. This leads to a poor end user experience.
No infrastructure is present to allow network transparent media handling. A distributed MPEG encoder will typically duplicate the same encoder algorithms found in a non-distributed encoder.
No provisions have been made for use by and use of technologies such as the GNOME desktop platform. Because the wheel is re-invented all the time, it's hard to properly integrate multimedia into the bigger whole of user's environment.
The GStreamer core does not use network transparent technologies at the lowest level as it only adds overhead for the local case. That said, it shouldn't be hard to create a wrapper around the core components. There are tcp plugins now that implement a GStreamer Data Protocol that allows pipelines to be split over TCP. These are located in the gst-plugins module directory gst/tcp.
We need solid media handling if we want to see Linux succeed on the desktop.
We must clear the road for commercially backed codecs and multimedia applications so that Linux can become an option for doing multimedia.