This is a summary of what GStreamer 1.0 brings in terms of new features and capabilities for users, application and plugin writers and platforms.
This document will not go over the list of deprecated/modified APIs and other cleanups that took place, people interested can check the porting document here.
Contents
As a user, why should I care about GStreamer 1.0 ?
The most noticeable change will be better performance in more situations, whether it is due to parts of GStreamer core that have been optimized for cpu usage and contention, or thanks to plugins being able to negotiate to more efficient data handling.
As a platform provider, why should I care about GStreamer 1.0 ?
All the items mentionned in the changes in GStreamer 1.0 for plugins were targetted from the beginning to make better usage of libraries and hardware that GStreamer elements wrap.
Plugins wrapping hardware/platform features can interact with all existing GStreamer plugins due to clear separation between GstCaps and how memory is stored.
Plugins wrapping hardware/platform features can better communicate with each other in a non-intrusive way.
Memory allocation is clearly controlled
Re-usability of existing code
Do I have to rewrite my whole application ?
While a lot of changes have gone into GStreamer 1.0, all the fundamentals of GStreamer (pipelines, elements, graphs, buffers, events, messages,. ..) are still present and the application-facing API has not changed much since most improvements were mostly required at the lower levels.
The application-facing changes are in fact quite minimalistic and targetted. Applications making use of gst_launch_parse() or convenience elements like playbin2 will find it quite trivial to port their application to 1.0.
Applications doing a bit more in-depth usage of GStreamer will have to check the porting document for changed APIs, the most notable one is the pad blocking and probes which have been merged together, easing addition and removal of elements in a pipeline on the fly.
As a plugin writer, what does Gstreamer 1.0 bring ?
Just as for Application writer, most of the concepts from GStreamer 0.10 for plugins remain the same (dataflow, buffers, graphs, pads, links). Most of the API changes can be converted by following the porting document.
But while GStreamer 0.10 managed to cover a wide-range of plugins and use-cases, some limitations in negotiation and data handling meant processing pipelines could not be as efficient as possible. This is clarified in 1.0.
Split how the data is stored from what it describes
In 0.10, the caps describing a data stream wer tightly coupled to how it was stored. For example video/x-raw-{yuv|rgb} implied the buffer data was in main memory, meaning new caps had to be created if the data was stored elsewhere breaking compatibility with existing video plugins. Yet another example would be that if you specified I420 yuv in the caps, it would require a very specific layout/stride/padding of the data in memory, requiring extra memory copies if the wrapped library or hardware didn't use that specific padding.
In 1.0, the caps now only describe what the data is. Whether your video data is on main memory or not, whether it has specific striding, the planes are contained in different memory banks, etc ... your caps will be video/x-raw.
This allows your plugins to interact with any other plugins handling the same caps, increasing re-usability.
Controlled memory access
This introduced the need for splitting out memory access, allowing plugins to access that memory and controlling access to that memory.
This is done through the following new concepts:
Mapping
All read and write access to memory need to be done through 'mapping' methods, where the caller can specify what usage it wishes to do of the memory, and where the provider of the memory can implement custom mapping, allocation, freeing, copying methods.
Allows plugins to provide data that can be read or writen by any other plugin without them knowing about how data is stored or where.
Allows for systems that require it, to ensure data is properly flushed back to the target memory banks when modifications happened.
Ensures that any allocation, freeing, copying called from any other plugin on that memory is done in the most efficient way possible. Plugins can also specify that memory is not writable or cannot be shared if needed.
GstMemory and GstAllocator
In order to achieve all the operations describe just above, buffers no longer hold a direct pointer to some memory location, but instead rely on a list of intermediate descriptor of the memory (GstMemory) which themselves point to a shared table of memory functions (GstAllocator).
Separate GstBuffer object/API from how data is stored. Users of the GstBuffer API don't have to care about those details, meaning GstBuffers are usable in any plugins, regardless of where data is stored and how.
Allows buffers in 1.0 to handle use-cases where data is stored in multiple locations, like for example multiple video memory banks.
Buffer metadata
GStreamer 1.0 adds the possibility to add metadata to GstBuffers : GstMeta.
A GstMeta is a simple structure that can be added to a GstBuffer. Usage of that specific GstMeta therefore requires having access to the header file at compile time.
This is done in such a way that buffers not knowing how to handle a certain GstMeta can safely ignore them.
Those metadata are registered with some initialization and freeing methods.
Metadata containing allocated information can be properly freed when the buffer gets finalized.
field-specific API
An example of GstMeta is the GstVideoMeta, containing video-specific metadata about the buffer, like the format, width, height, number of planes, location of those planes, stride. Plugins can then use this video metadata to transfer video information about the buffer that wasn't available coherently in GStreamer 0.10, like strides.
Other examples of GstMeta that could be created would include hardware-specific information, like a context, which would enable two plugins understanding that GstMeta to make methods calls for more efficient processing even though they are separated by several other elements that don't understand that GstMeta and the application knows nothing about it.
Plugins can add usage-specific GstMeta to buffers, enabling them to have a better understanding of the data they are handling and making usage of more efficient methods.
The GstMeta structure can contain any kind of fields, including pointers and function pointers, which enables creation of new inter-plugin APIs.
For example, the GstVideoMeta also contains two function pointers for mapping and unmapping a video plane. This allows plugins that have got data stored in separate memory regions to implement specific GstVideoMeta map/unmap methods to provide access to the plane in an efficient way without having to concatenate all planes together (default behaviour with gst_buffer_map).
Due to also containing function pointers, GstMeta can also act as an extra cross-plugin API.
GstMeta allows adding domain-specific portable API, while keeping backward compatibility for plugins who don't know how to handle it.
GstMeta helps solve the problem by which efficient processing that needs to be achieved by having better communication/visibility between elements can be done without disrupting other elements.
Control memory allocation
In order to achieve the best performance, plugins need to be able to control the allocation of data that will be used downstream.
In GStreamer 0.10, plugins would request a buffer from downstream elements in which they would write the data. While this avoided memory copies in some situations, the control over what kind of memory was requested and how it would be used was very poor.
In addition, requesting a new buffer would be slow (since the call needed to go all the way to the provider of buffers, which could be many elements downstream) and the control over the number of buffers available could not be controlled.
One new way to handle this in Gstreamer 1.0 is that elements can provide a GstBufferPool, which is an object providing buffers. That pool can be shared accross multiple objects. That pool can also be configured by the element using it (which might not be the one providing the pool) such as for minimum as maximum number of buffers available, or for specific memory alignment.
If a minimum number is specified, the pool will pre-allocate that number of buffers resulting in fast availability of buffers when needed.
If a maximum number is specified (to cope with certain memory constraints for examples), then request to the pool for new buffers will block until a buffer is made available.
Requesting optimal buffers is now faster and more controlled via usage of the GstBufferPool.
Buffers provided by a GstBufferPool are recycled when no longer used, resulting in more efficient memory usage (avoids free/alloc cycle).
Users of buffers can configure pools for minimum/maximum numbers of buffers needed. Users can better control their memory constraints.
Delayed processing, knowing more about downstream
Controlling allocation in 1.0 is now done via a ALLOCATION query that any element can send downstream, asking what allocation methods are available and whether it wishes to be given a GstBufferPool it can use for its output.
Elements can request a GstBufferPool for their output, which is a faster local object from which it can acquire buffers to write its output.
Every element responding to that query (including the provider of the GstBufferPool) also indicates which GstMeta it understands and handles.
An example would be for a videosink to indicate it can handle the GstVideoMeta, the caller then knows it can set that meta on the buffer it outputs and know that downstream elements will handle it.
Another example is the GstVideoCropMeta, which contains information about cropping. If an element along the way (or the sink) indicates it handles that GstMeta, the decoder (or videocrop element!) will be able to avoid doing the actual cropping (which might include expensive memory copies) and instead just specify the cropping region in the GstVideoCropMeta.
The ALLOCATION query allows elements to have a better visibility on what kind of GstMeta downstream elements can handle.
The ALLOCATION query allows elements to inform upstream elements on what kind of GstMeta it supports.
Elements can delay processing downstream where it can be more efficiently done.