gstadaptivedemux

What is an adaptive demuxer? Adaptive demuxers are special demuxers in the sense that they don't actually demux data received from upstream but download the data themselves.

Adaptive formats (HLS, DASH, MSS) are composed of a manifest file and a set of fragments. The manifest describes the available media and the sequence of fragments to use. Each fragment contains a small part of the media (typically only a few seconds). It is possible for the manifest to have the same media available in different configurations (bitrates for example) so that the client can select the one that best suits its scenario (network fluctuation, hardware requirements...). It is possible to switch from one representation of the media to another during playback. That's why it is called 'adaptive', because it can be adapted to the client's needs.

Architectural overview: The manifest is received by the demuxer in its sink pad and, upon receiving EOS, it parses the manifest and exposes the streams available in it. For each stream a source element will be created and will download the list of fragments one by one. Once a fragment is finished downloading, the next URI is set to the source element and it starts fetching it and pushing through the stream's pad. This implies that each stream is independent from each other as it runs on a separate thread.

After downloading each fragment, the download rate of it is calculated and the demuxer has a chance to switch to a different bitrate if needed. The switch can be done by simply pushing a new caps before the next fragment when codecs are the same, or by exposing a new pad group if it needs a codec change.

Extra features:

  • Not linked streams: Streams that are not-linked have their download threads interrupted to save network bandwidth. When they are relinked a reconfigure event is received and the stream is restarted.

Subclasses: While GstAdaptiveDemux is responsible for the workflow, it knows nothing about the intrinsics of the subclass formats, so the subclasses are responsible for maintaining the manifest data structures and stream information.

Functions

gst_adaptive_demux_find_stream_for_pad

GstAdaptiveDemuxStream *
gst_adaptive_demux_find_stream_for_pad (GstAdaptiveDemux * demux,
                                        GstPad * pad)

Parameters:

demux
No description available
pad
No description available
Returns ( GstAdaptiveDemuxStream * ) –
No description available

gst_adaptive_demux_get_client_now_utc

GDateTime *
gst_adaptive_demux_get_client_now_utc (GstAdaptiveDemux * demux)

Used to find the client's estimate of UTC, using the system realtime clock.

Parameters:

demux

GstAdaptiveDemux Returns: the client's estimate of UTC

Returns ( GDateTime * ) –
No description available

gst_adaptive_demux_get_monotonic_time

GstClockTime
gst_adaptive_demux_get_monotonic_time (GstAdaptiveDemux * demux)

Parameters:

demux
No description available
Returns ( GstClockTime ) –

a monotonically increasing time, using the system realtime clock


gst_adaptive_demux_get_qos_earliest_time

GstClockTime
gst_adaptive_demux_get_qos_earliest_time (GstAdaptiveDemux * demux)

Parameters:

demux
No description available
Returns ( GstClockTime ) –

The QOS earliest time

Since : 1.20


gst_adaptive_demux_get_type

GType
gst_adaptive_demux_get_type ()
Returns ( GType ) –
No description available

gst_adaptive_demux_is_running

gboolean
gst_adaptive_demux_is_running (GstAdaptiveDemux * demux)

Returns FALSE if shutdown has started (transitioning down from PAUSED), otherwise TRUE.

Parameters:

demux

GstAdaptiveDemux Returns: whether the demuxer is processing data

Returns ( gboolean ) –
No description available

gst_adaptive_demux_set_stream_struct_size

void 
gst_adaptive_demux_set_stream_struct_size (GstAdaptiveDemux * demux,
                                           gsize struct_size)

Parameters:

demux
No description available
struct_size
No description available
Returns ( void ) –
No description available

gst_adaptive_demux_stream_advance_fragment

GstFlowReturn
gst_adaptive_demux_stream_advance_fragment (GstAdaptiveDemux * demux,
                                            GstAdaptiveDemuxStream * stream,
                                            GstClockTime duration)

Parameters:

demux
No description available
stream
No description available
duration
No description available
Returns ( GstFlowReturn ) –
No description available

gst_adaptive_demux_stream_fragment_clear

void 
gst_adaptive_demux_stream_fragment_clear (GstAdaptiveDemuxStreamFragment * f)

Parameters:

f
No description available
Returns ( void ) –
No description available

gst_adaptive_demux_stream_new

GstAdaptiveDemuxStream *
gst_adaptive_demux_stream_new (GstAdaptiveDemux * demux,
                               GstPad * pad)

Parameters:

demux
No description available
pad
No description available
Returns ( GstAdaptiveDemuxStream * ) –
No description available

gst_adaptive_demux_stream_push_buffer

GstFlowReturn
gst_adaptive_demux_stream_push_buffer (GstAdaptiveDemuxStream * stream,
                                       GstBuffer * buffer)

Parameters:

stream
No description available
buffer
No description available
Returns ( GstFlowReturn ) –
No description available

gst_adaptive_demux_stream_queue_event

void 
gst_adaptive_demux_stream_queue_event (GstAdaptiveDemuxStream * stream,
                                       GstEvent * event)

Parameters:

stream
No description available
event
No description available
Returns ( void ) –
No description available

gst_adaptive_demux_stream_set_caps

void 
gst_adaptive_demux_stream_set_caps (GstAdaptiveDemuxStream * stream,
                                    GstCaps * caps)

Parameters:

stream
No description available
caps
No description available
Returns ( void ) –
No description available

gst_adaptive_demux_stream_set_tags

void 
gst_adaptive_demux_stream_set_tags (GstAdaptiveDemuxStream * stream,
                                    GstTagList * tags)

Parameters:

stream
No description available
tags
No description available
Returns ( void ) –
No description available

Structures

GstAdaptiveDemux

The opaque GstAdaptiveDemux data structure.


GstAdaptiveDemuxClass

struct _GstAdaptiveDemuxClass
{

  /**
   * process_manifest: Parse the manifest
   * @demux: #GstAdaptiveDemux
   * @manifest: the manifest to be parsed
   *
   * Parse the manifest and add the created streams using
   * gst_adaptive_demux_stream_new()
   *
   * Returns: %TRUE if successful
   */
  gboolean      (*process_manifest) (GstAdaptiveDemux * demux, GstBuffer * manifest);

  /**
   * get_manifest_update_interval:
   * @demux: #GstAdaptiveDemux
   *
   * Used during live streaming, the subclass should return the interval
   * between successive manifest updates
   *
   * Returns: the update interval in microseconds
   */
  gint64        (*get_manifest_update_interval) (GstAdaptiveDemux * demux);

  /**
   * update_manifest:
   * @demux: #GstAdaptiveDemux
   *
   * During live streaming, this will be called for the subclass to update its
   * manifest with the new version. By default it fetches the manifest URI
   * and passes it to GstAdaptiveDemux::update_manifest_data().
   *
   * Returns: #GST_FLOW_OK is all succeeded, #GST_FLOW_EOS if the stream ended
   *          or #GST_FLOW_ERROR if an error happened
   */
  GstFlowReturn (*update_manifest) (GstAdaptiveDemux * demux);

  /**
   * update_manifest_data:
   * @demux: #GstAdaptiveDemux
   * @buf: Downloaded manifest data
   *
   * During live streaming, this will be called for the subclass to update its
   * manifest with the new version
   *
   * Returns: #GST_FLOW_OK is all succeeded, #GST_FLOW_EOS if the stream ended
   *          or #GST_FLOW_ERROR if an error happened
   */
  GstFlowReturn (*update_manifest_data) (GstAdaptiveDemux * demux, GstBuffer * buf);

  gboolean      (*is_live)          (GstAdaptiveDemux * demux);
  GstClockTime  (*get_duration)     (GstAdaptiveDemux * demux);

  /**
   * reset:
   * @demux: #GstAdaptiveDemux
   *
   * Reset the internal state of the subclass, getting ready to restart with
   * a new stream afterwards
   */
  void          (*reset)            (GstAdaptiveDemux * demux);

  /**
   * seek:
   * @demux: #GstAdaptiveDemux
   * @seek: a seek #GstEvent
   *
   * The demuxer should seek on all its streams to the specified position
   * in the seek event
   *
   * Returns: %TRUE if successful
   */
  gboolean      (*seek)             (GstAdaptiveDemux * demux, GstEvent * seek);

  /**
   * has_next_period:
   * @demux: #GstAdaptiveDemux
   *
   * Checks if there is a next period following the current one.
   * DASH can have multiple medias chained in its manifest, when one finishes
   * this function is called to verify if there is a new period to be played
   * in sequence.
   *
   * Returns: %TRUE if there is another period
   */
  gboolean      (*has_next_period)  (GstAdaptiveDemux * demux);
  /**
   * advance_period:
   * @demux: #GstAdaptiveDemux
   *
   * Advances the manifest to the next period. New streams should be created
   * using gst_adaptive_demux_stream_new().
   */
  void          (*advance_period)  (GstAdaptiveDemux * demux);

  void          (*stream_free)     (GstAdaptiveDemuxStream * stream);
  GstFlowReturn (*stream_seek)     (GstAdaptiveDemuxStream * stream, gboolean forward, GstSeekFlags flags, GstClockTime target_ts, GstClockTime * final_ts);
  gboolean      (*stream_has_next_fragment)  (GstAdaptiveDemuxStream * stream);
  GstFlowReturn (*stream_advance_fragment) (GstAdaptiveDemuxStream * stream);

  /**
   * need_another_chunk:
   * @stream: #GstAdaptiveDemuxStream
   *
   * If chunked downloading is used (chunk_size != 0) this is called once a
   * chunk is finished to decide whether more has to be downloaded or not.
   * May update chunk_size to a different value
   */
  gboolean      (*need_another_chunk) (GstAdaptiveDemuxStream * stream);

  /**
   * stream_update_fragment_info:
   * @stream: #GstAdaptiveDemuxStream
   *
   * Requests the stream to set the information about the current fragment to its
   * current fragment struct
   *
   * Returns: #GST_FLOW_OK in success, #GST_FLOW_ERROR on error and #GST_FLOW_EOS
   *          if there is no fragment.
   */
  GstFlowReturn (*stream_update_fragment_info) (GstAdaptiveDemuxStream * stream);
  /**
   * stream_select_bitrate:
   * @stream: #GstAdaptiveDemuxStream
   * @bitrate: the bitrate to select (in bytes per second)
   *
   * The stream should try to select the bitrate that is the greater, but not
   * greater than the requested bitrate. If it needs a codec change it should
   * create the new stream using gst_adaptive_demux_stream_new(). If it only
   * needs a caps change it should set the new caps using
   * gst_adaptive_demux_stream_set_caps().
   *
   * Returns: %TRUE if the stream changed bitrate, %FALSE otherwise
   */
  gboolean      (*stream_select_bitrate) (GstAdaptiveDemuxStream * stream, guint64 bitrate);
  /**
   * stream_get_fragment_waiting_time:
   * @stream: #GstAdaptiveDemuxStream
   *
   * For live streams, requests how much time should be waited before starting
   * to download the fragment. This is useful to avoid downloading a fragment that
   * isn't available yet.
   *
   * Returns: The waiting time in microseconds
   */
  gint64        (*stream_get_fragment_waiting_time) (GstAdaptiveDemuxStream * stream);

  /**
   * start_fragment:
   * @demux: #GstAdaptiveDemux
   * @stream: #GstAdaptiveDemuxStream
   *
   * Notifies the subclass that the given stream is starting the download
   * of a new fragment. Can be used to reset/init internal state that is
   * needed before each fragment, like decryption engines.
   *
   * Returns: %TRUE if successful.
   */
  gboolean      (*start_fragment) (GstAdaptiveDemux * demux, GstAdaptiveDemuxStream * stream);
  /**
   * finish_fragment:
   * @demux: #GstAdaptiveDemux
   * @stream: #GstAdaptiveDemuxStream
   *
   * Notifies the subclass that a fragment download was finished.
   * It can be used to cleanup internal state after a fragment and
   * also push any pending data before moving to the next fragment.
   */
  GstFlowReturn (*finish_fragment) (GstAdaptiveDemux * demux, GstAdaptiveDemuxStream * stream);
  /**
   * data_received:
   * @demux: #GstAdaptiveDemux
   * @stream: #GstAdaptiveDemuxStream
   * @buffer: #GstBuffer
   *
   * Notifies the subclass that a fragment chunk was downloaded. The subclass
   * can look at the data and modify/push data as desired.
   *
   * Returns: #GST_FLOW_OK if successful, #GST_FLOW_ERROR in case of error.
   */
  GstFlowReturn (*data_received) (GstAdaptiveDemux * demux, GstAdaptiveDemuxStream * stream, GstBuffer * buffer);

  /**
   * get_live_seek_range:
   * @demux: #GstAdaptiveDemux
   * @start: pointer to put the start position allowed to seek to
   * @stop: pointer to put the stop position allowed to seek to
   *
   * Gets the allowed seek start and stop positions for the current live stream
   *
   * Return: %TRUE if successful
   */
  gboolean (*get_live_seek_range) (GstAdaptiveDemux * demux, gint64 * start, gint64 * stop);

  /**
   * get_presentation_offset:
   * @demux: #GstAdaptiveDemux
   * @stream: #GstAdaptiveDemuxStream
   *
   * Gets the delay to apply to @stream.
   *
   * Return: a #GstClockTime representing the (positive) time offset to apply to
   * @stream.
   */
  GstClockTime (*get_presentation_offset) (GstAdaptiveDemux *demux, GstAdaptiveDemuxStream *stream);

  /**
   * get_period_start_time:
   * @demux: #GstAdaptiveDemux
   *
   * Gets the start time of the current period. Timestamps are resetting to 0
   * after each period but we have to maintain a continuous stream and running
   * time so need to know the start time of the current period.
   *
   * Return: a #GstClockTime representing the start time of the currently
   * selected period.
   */
  GstClockTime (*get_period_start_time) (GstAdaptiveDemux *demux);

  /**
   * requires_periodical_playlist_update:
   * @demux: #GstAdaptiveDemux
   *
   * Some adaptive streaming protocols allow the client to download
   * the playlist once and build up the fragment list based on the
   * current fragment metadata. For those protocols the demuxer
   * doesn't need to periodically refresh the playlist. This vfunc
   * is relevant only for live playback scenarios.
   *
   * Return: %TRUE if the playlist needs to be refreshed periodically by the demuxer.
   */
  gboolean (*requires_periodical_playlist_update) (GstAdaptiveDemux * demux);
};

Fields

process_manifest () –
No description available
get_manifest_update_interval () –
No description available
update_manifest () –
No description available
update_manifest_data () –
No description available
is_live () –
No description available
get_duration () –
No description available
reset () –
No description available
seek () –
No description available
has_next_period () –
No description available
advance_period () –
No description available
stream_free () –
No description available
stream_seek () –
No description available
stream_has_next_fragment () –
No description available
stream_advance_fragment () –
No description available
need_another_chunk () –
No description available
stream_update_fragment_info () –
No description available
stream_select_bitrate () –
No description available
stream_get_fragment_waiting_time () –
No description available
start_fragment () –
No description available
finish_fragment () –
No description available
data_received () –
No description available
get_live_seek_range () –
No description available
get_presentation_offset () –
No description available
get_period_start_time () –
No description available
requires_periodical_playlist_update () –
No description available

GstAdaptiveDemuxStream

struct _GstAdaptiveDemuxStream
{
  GstPad *pad;
  GstPad *internal_pad;

  GstAdaptiveDemux *demux;

  GstSegment segment;

  GstCaps *pending_caps;
  GstEvent *pending_segment;
  GstTagList *pending_tags;
  gboolean need_header;
  GList *pending_events;

  GstFlowReturn last_ret;
  GError *last_error;

  GstTask *download_task;
  GRecMutex download_lock;

  gboolean restart_download;
  gboolean discont;

  gboolean downloading_first_buffer;
  gboolean downloading_header;
  gboolean downloading_index;

  gboolean bitrate_changed;

  /* download tooling */
  GstElement *src;
  guint last_status_code;
  GstPad *src_srcpad;
  GstElement *uri_handler;
  GstElement *queue;
  GMutex fragment_download_lock;
  GCond fragment_download_cond;
  gboolean download_finished;   /* protected by fragment_download_lock */
  gboolean cancelled; /* protected by fragment_download_lock */
  gboolean replaced; /* replaced in a bitrate switch (used with cancelled) */
  gboolean src_at_ready;     /* protected by fragment_download_lock */
  gboolean starting_fragment;
  gboolean first_fragment_buffer;
  gint64 download_start_time;
  gint64 download_total_bytes;
  guint64 current_download_rate;

  /* amount of data downloaded in current fragment (pre-queue2) */
  guint64 fragment_bytes_downloaded;
  /* bitrate of the previous fragment (pre-queue2) */
  guint64 last_bitrate;
  /* latency (request to first byte) and full download time (request to EOS)
   * of previous fragment (pre-queue2) */
  GstClockTime last_latency;
  GstClockTime last_download_time;

  /* Average for the last fragments */
  guint64 moving_bitrate;
  guint moving_index;
  guint64 *fragment_bitrates;

  /* QoS data : UNUSED !!! */
  GstClockTime qos_earliest_time;

  GstAdaptiveDemuxStreamFragment fragment;

  guint download_error_count;

  /* TODO check if used */
  gboolean eos;

  gboolean do_block; /* TRUE if stream should block on preroll */
};

Fields

pad (GstPad *) –
No description available
internal_pad (GstPad *) –
No description available
demux (GstAdaptiveDemux *) –
No description available
segment (GstSegment) –
No description available
pending_caps (GstCaps *) –
No description available
pending_segment (GstEvent *) –
No description available
pending_tags (GstTagList *) –
No description available
need_header (gboolean) –
No description available
pending_events (GList *) –
No description available
last_ret (GstFlowReturn) –
No description available
last_error (GError *) –
No description available
download_task (GstTask *) –
No description available
download_lock (GRecMutex) –
No description available
restart_download (gboolean) –
No description available
discont (gboolean) –
No description available
downloading_first_buffer (gboolean) –
No description available
downloading_header (gboolean) –
No description available
downloading_index (gboolean) –
No description available
bitrate_changed (gboolean) –
No description available
src (GstElement *) –
No description available
last_status_code (guint) –
No description available
src_srcpad (GstPad *) –
No description available
uri_handler (GstElement *) –
No description available
queue (GstElement *) –
No description available
fragment_download_lock (GMutex) –
No description available
fragment_download_cond (GCond) –
No description available
download_finished (gboolean) –
No description available
cancelled (gboolean) –
No description available
replaced (gboolean) –
No description available
src_at_ready (gboolean) –
No description available
starting_fragment (gboolean) –
No description available
first_fragment_buffer (gboolean) –
No description available
download_start_time (gint64) –
No description available
download_total_bytes (gint64) –
No description available
current_download_rate (guint64) –
No description available
fragment_bytes_downloaded (guint64) –
No description available
last_bitrate (guint64) –
No description available
last_latency (GstClockTime) –
No description available
last_download_time (GstClockTime) –
No description available
moving_bitrate (guint64) –
No description available
moving_index (guint) –
No description available
fragment_bitrates (guint64 *) –
No description available
qos_earliest_time (GstClockTime) –
No description available
No description available
download_error_count (guint) –
No description available
eos (gboolean) –
No description available
do_block (gboolean) –
No description available

GstAdaptiveDemuxStreamFragment

struct _GstAdaptiveDemuxStreamFragment
{
  GstClockTime timestamp;
  GstClockTime duration;

  gchar *uri;
  gint64 range_start;
  gint64 range_end;

  /* when chunked downloading is used, may be be updated need_another_chunk() */
  guint chunk_size;

  /* when headers are needed */
  gchar *header_uri;
  gint64 header_range_start;
  gint64 header_range_end;

  /* when index is needed */
  gchar *index_uri;
  gint64 index_range_start;
  gint64 index_range_end;

  /* Nominal bitrate as provided by
   * sub-class or calculated by base-class */
  guint bitrate;

  gboolean finished;
};

Fields

timestamp (GstClockTime) –
No description available
duration (GstClockTime) –
No description available
uri (gchar *) –
No description available
range_start (gint64) –
No description available
range_end (gint64) –
No description available
chunk_size (guint) –
No description available
header_uri (gchar *) –
No description available
header_range_start (gint64) –
No description available
header_range_end (gint64) –
No description available
index_uri (gchar *) –
No description available
index_range_start (gint64) –
No description available
index_range_end (gint64) –
No description available
bitrate (guint) –
No description available
finished (gboolean) –
No description available

The results of the search are