All streams of data in GStreamer are chopped up into chunks that are passed from a source pad on one element to a sink pad on another element. GstMiniObject is the structure used to hold these chunks of data.
GstMiniObject contains the following important types:
An exact type indicating what type of data (event, buffer, ...) this GstMiniObject is.
A reference count indicating the number of elements currently holding a reference to the miniobject. When the reference count falls to zero, the miniobject will be disposed, and its memory will be freed in some sense (see below for more details).
For data transport, there are two types of GstMiniObject defined: events (control) and buffers (content).
Buffers may contain any sort of data that the two linked pads know how to handle. Normally, a buffer contains a chunk of some sort of audio or video data that flows from one element to another.
Buffers also contain metadata describing the buffer's contents. Some of the important types of metadata are:
Pointers to one or more GstMemory objects. GstMemory objects are refcounted objects that encapsulate a region of memory.
A timestamp indicating the preferred display timestamp of the content in the buffer.
Events contain information on the state of the stream flowing between the two linked pads. Events will only be sent if the element explicitly supports them, else the core will (try to) handle the events automatically. Events are used to indicate, for example, a media type, the end of a media stream or that the cache should be flushed.
Events may contain several of the following items:
A subtype indicating the type of the contained event.
The other contents of the event depend on the specific event type.
Events will be discussed extensively in Chapter 17, Events: Seeking, Navigation and More. Until then, the only event that will be used is the EOS event, which is used to indicate the end-of-stream (usually end-of-file).
Buffers are able to store chunks of memory of several different types. The most generic type of buffer contains memory allocated by malloc(). Such buffers, although convenient, are not always very fast, since data often needs to be specifically copied into the buffer.
Many specialized elements create buffers that point to special memory. For example, the filesrc element usually maps a file into the address space of the application (using mmap()), and creates buffers that point into that address range. These buffers created by filesrc act exactly like generic buffers, except that they are read-only. The buffer freeing code automatically determines the correct method of freeing the underlying memory. Downstream elements that receive these kinds of buffers do not need to do anything special to handle or unreference it.
Another way an element might get specialized buffers is to request them from a downstream peer through a GstBufferPool or GstAllocator. Elements can ask a GstBufferPool or GstAllocator from the downstream peer element. If downstream is able to provide these objects, upstream can use them to allocate buffers. See more in Memory allocation.
Many sink elements have accelerated methods for copying data to hardware, or have direct access to hardware. It is common for these elements to be able to create a GstBufferPool or GstAllocator for their upstream peers. One such example is ximagesink. It creates buffers that contain XImages. Thus, when an upstream peer copies data into the buffer, it is copying directly into the XImage, enabling ximagesink to draw the image directly to the screen instead of having to copy data into an XImage first.
Filter elements often have the opportunity to either work on a buffer in-place, or work while copying from a source buffer to a destination buffer. It is optimal to implement both algorithms, since the GStreamer framework can choose the fastest algorithm as appropriate. Naturally, this only makes sense for strict filters -- elements that have exactly the same format on source and sink pads.