When playing complex media, each sound and video sample must be played in a specific order at a specific time. For this purpose, GStreamer provides a synchronization mechanism.
GStreamer provides support for the following use cases:
Non-live sources with access faster than playback rate. This is the case where one is reading media from a file and playing it back in a synchronized fashion. In this case, multiple streams need to be synchronized, like audio, video and subtitles.
Capture and synchronized muxing/mixing of media from multiple live sources. This is a typical use case where you record audio and video from a microphone/camera and mux it into a file for storage.
Streaming from (slow) network streams with buffering. This is the typical web streaming case where you access content from a streaming server with http.
Capture from live source and and playback to live source with configurable latency. This is used when, for example, capture from a camera, apply an effect and display the result. It is also used when streaming low latency content over a network with UDP.
Simultaneous live capture and playback from prerecorded content. This is used in audio recording cases where you play a previously recorded audio and record new samples, the purpose is to have the new audio perfectly in sync with the previously recorded data.
GStreamer uses a
GstClock object, buffer
timestamps and a SEGMENT event to synchronize streams in a pipeline
as we will see in the next sections.
In a typical computer, there are many sources that can be used as a
time source, e.g., the system time, soundcards, CPU performance
counters, ... For this reason, there are many
GstClock implementations available in GStreamer.
The clock time doesn't always start from 0 or from some known value.
Some clocks start counting from some known start date, other clocks start
counting since last reboot, etc...
GstClock returns the
according to that clock with
The absolute-time (or clock time) of a clock is monotonically increasing.
From the absolute-time is a running-time
calculated, which is simply the difference between a previous snapshot
of the absolute-time called the base-time.
running-time = absolute-time - base-time
GstPipeline object maintains a
GstClock object and a base-time when it goes
to the PLAYING state. The pipeline gives a handle to the selected
GstClock to each element in the pipeline along
with selected base-time. The pipeline will select a base-time in such
a way that the running-time reflects the total time spent in the
PLAYING state. As a result, when the pipeline is PAUSED, the
running-time stands still.
Because all objects in the pipeline have the same clock and base-time, they can thus all calculate the running-time according to the pipeline clock.