webrtcsink

webrtcsink

All-batteries included GStreamer WebRTC producer, that tries its best to do The Right Thing™.

Use case

The webrtcbin element in GStreamer is extremely flexible and powerful, but using it can be a difficult exercise. When all you want to do is serve a fixed set of streams to any number of consumers, webrtcsink (which wraps webrtcbin internally) can be a useful alternative.

Features

webrtcsink implements the following features:

  • Built-in signaller: when using the default signalling server, this element will perform signalling without requiring application interaction. This makes it usable directly from gst-launch.

  • Application-provided signalling: webrtcsink can be instantiated by an application with a custom signaller. That signaller must be a GObject, and must implement the Signallable interface as defined here. The default signaller can be used as an example.

    An example project is also available to use as a boilerplate for implementing and using a custom signaller.

  • Sandboxed consumers: when a consumer is added, its encoder / payloader / webrtcbin elements run in a separately managed pipeline. This provides a certain level of sandboxing, as opposed to having those elements running inside the element itself.

    It is important to note that at this moment, encoding is not shared between consumers. While this is not on the roadmap at the moment, nothing in the design prevents implementing this optimization.

  • Congestion control: the element leverages transport-wide congestion control feedback messages in order to adapt the bitrate of individual consumers' video encoders to the available bandwidth.

  • Configuration: the level of user control over the element is slowly expanding, consult gst-inspect-1.0 for more information on the available properties and signals.

  • Packet loss mitigation: webrtcsink now supports sending protection packets for Forward Error Correction, modulating the amount as a function of the available bandwidth, and can honor retransmission requests. Both features can be disabled via properties.

It is important to note that full control over the individual elements used by webrtcsink is not on the roadmap, as it will act as a black box in that respect, for example webrtcsink wants to reserve control over the bitrate for congestion control.

A signal is now available however for the application to provide the initial configuration for the encoders webrtcsink instantiates.

If more granular control is required, applications should use webrtcbin directly, webrtcsink will focus on trying to just do the right thing, although it might expose more interfaces to guide and tune the heuristics it employs.

Building

Make sure to install the development packages for some codec libraries beforehand, such as libx264, libvpx and libopusenc, exact names depend on your distribution.

cargo build

Usage

Open three terminals. In the first, run:

WEBRTCSINK_SIGNALLING_SERVER_LOG=debug cargo run --bin gst-webrtc-signalling-server

In the second, run:

python3 -m http.server -d www/

In the third, run:

export GST_PLUGIN_PATH=$PWD/target/debug:$GST_PLUGIN_PATH
gst-launch-1.0 webrtcsink name=ws videotestsrc ! ws. audiotestsrc ! ws.

When the pipeline above is running successfully, open a browser and point it to the http server:

gio open http://127.0.0.1:8000

You should see an identifier listed in the left-hand panel, click on it. You should see a test video stream, and hear a test tone.

Configuration

The element itself can be configured through its properties, see gst-inspect-1.0 webrtcsink for more information about that, in addition the default signaller also exposes properties for configuring it, in particular setting the signalling server address, those properties can be accessed through the gst::ChildProxy interface, for example with gst-launch:

gst-launch-1.0 webrtcsink signaller::address="ws://127.0.0.1:8443" ..

Enable 'navigation' a.k.a user interactivity with the content

webrtcsink implements the GstNavigation interface which allows interacting with the content, for example move with your mouse, entering keys with the keyboard, etc... On top of that a WebRTCDataChannel based protocol has been implemented and can be activated with the enable-data-channel-navigation=true property. The demo implements the protocol and you can easily test this feature, using the wpesrc for example.

As an example, the following pipeline allows you to navigate the GStreamer documentation inside the video running within your web browser (in http://127.0.0.1:8000 if you followed previous steps of that readme):

gst-launch-1.0 wpesrc location=https://gstreamer.freedesktop.org/documentation/ ! webrtcsink enable-data-channel-navigation=true

Testing congestion control

For the purpose of testing congestion in a reproducible manner, a simple tool has been used, I only used it on Linux but it is documented as usable on MacOS too. I had to run the client browser on a separate machine on my local network for congestion to actually be applied, I didn't look into why that was necessary.

My testing procedure was:

  • identify the server machine network interface (eg with ifconfig on Linux)

  • identify the client machine IP address (eg with ifconfig on Linux)

  • start the various services as explained in the Usage section (use GST_DEBUG=webrtcsink:7 to get detailed logs about congestion control)

  • start playback in the client browser

  • Run a comcast command on the server machine, for instance:

    $HOME/go/bin/comcast --device=$SERVER_INTERFACE --target-bw 3000 --target-addr=$CLIENT_IP --target-port=1:65535 --target-proto=udp
    
  • Observe the bitrate sharply decreasing, playback should slow down briefly then catch back up

  • Remove the bandwidth limitation, and observe the bitrate eventually increasing back to a maximum:

    $HOME/go/bin/comcast --device=$SERVER_INTERFACE --stop
    

For comparison, the congestion control property can be set to disabled on webrtcsink, then the above procedure applied again, the expected result is for playback to simply crawl down to a halt until the bandwidth limitation is lifted:

gst-launch-1.0 webrtcsink congestion-control=disabled

Monitoring tool

An example server / client application for monitoring per-consumer stats can be found here.

Hierarchy

GObject
    ╰──GInitiallyUnowned
        ╰──GstObject
            ╰──GstElement
                ╰──GstBin
                    ╰──webrtcsink

Implemented interfaces

Factory details

Authors: – Mathieu Duponchelle

Classification:Sink/Network/WebRTC

Rank – none

Plugin – rswebrtc

Package – gst-plugin-webrtc

Pad Templates

audio_%u

audio/x-raw:

Presencerequest

Directionsink

Object typeGstPad


video_%u

video/x-raw:

video/x-raw(memory:CUDAMemory):

video/x-raw(memory:GLMemory):

video/x-raw(memory:NVMM):

Presencerequest

Directionsink

Object typeGstPad


Signals

consumer-added

consumer_added_callback (GstElement * param_0,
                         gchararray arg0,
                         GstElement * arg1,
                         gpointer udata)
def consumer_added_callback (param_0, arg0, arg1, udata):
    #python callback for the 'consumer-added' signal
function consumer_added_callback(param_0: GstElement * param_0, arg0: gchararray arg0, arg1: GstElement * arg1, udata: gpointer udata): {
    // javascript callback for the 'consumer-added' signal
}

Parameters:

param_0
No description available
arg0
No description available
arg1
No description available
udata
No description available

Flags: Run Last


consumer-removed

consumer_removed_callback (GstElement * param_0,
                           gchararray arg0,
                           GstElement * arg1,
                           gpointer udata)
def consumer_removed_callback (param_0, arg0, arg1, udata):
    #python callback for the 'consumer-removed' signal
function consumer_removed_callback(param_0: GstElement * param_0, arg0: gchararray arg0, arg1: GstElement * arg1, udata: gpointer udata): {
    // javascript callback for the 'consumer-removed' signal
}

Parameters:

param_0
No description available
arg0
No description available
arg1
No description available
udata
No description available

Flags: Run Last


encoder-setup

gboolean
encoder_setup_callback (GstElement * param_0,
                        gchararray arg0,
                        gchararray arg1,
                        GstElement * arg2,
                        gpointer udata)
def encoder_setup_callback (param_0, arg0, arg1, arg2, udata):
    #python callback for the 'encoder-setup' signal
function encoder_setup_callback(param_0: GstElement * param_0, arg0: gchararray arg0, arg1: gchararray arg1, arg2: GstElement * arg2, udata: gpointer udata): {
    // javascript callback for the 'encoder-setup' signal
}

Parameters:

param_0
No description available
arg0
No description available
arg1
No description available
arg2
No description available
udata
No description available
Returns (gboolean)
No description available

Flags: Run Last


Action Signals

get-sessions

g_signal_emit_by_name (param_0, "get-sessions", &ret);
ret = param_0.emit ("get-sessions")
let ret = param_0.emit ("get-sessions");

Parameters:

param_0 (GstElement *)
No description available
Returns (GStrv *)
No description available

Flags: Run Last / Action


Properties

audio-caps

“audio-caps” GstCaps *

Governs what audio codecs will be proposed

Flags : Read / Write

Default value :

audio/x-opus

congestion-control

“congestion-control” Web-rtcsink-congestion-control *

Defines how congestion is controlled, if at all

Flags : Read / Write

Default value : gcc (2)


do-fec

“do-fec” gboolean

Whether the element should negotiate and send FEC data

Flags : Read / Write

Default value : true


do-retransmission

“do-retransmission” gboolean

Whether the element should offer to honor retransmission requests

Flags : Read / Write

Default value : true


enable-data-channel-navigation

“enable-data-channel-navigation” gboolean

Enable navigation events through a dedicated WebRTCDataChannel

Flags : Read / Write

Default value : false


max-bitrate

“max-bitrate” guint

Minimal bitrate to use (in bit/sec) when computing it through the congestion control algorithm

Flags : Read / Write

Default value : 8192000


meta

“meta” GstStructure *

Free form metadata about the producer

Flags : Read / Write


min-bitrate

“min-bitrate” guint

Minimal bitrate to use (in bit/sec) when computing it through the congestion control algorithm

Flags : Read / Write

Default value : 1000


start-bitrate

“start-bitrate” guint

Start bitrate to use (in bit/sec)

Flags : Read / Write

Default value : 2048000


stats

“stats” GstStructure *

Statistics for the current consumers

Flags : Read

Default value :

application/x-webrtcsink-stats;

stun-server

“stun-server” gchararray

The STUN server of the form stun://hostname:port

Flags : Read / Write

Default value : stun://stun.l.google.com:19302


turn-servers

“turn-servers” GstValueArray *

The TURN servers of the form <"turn(s)://username:password@host:port", "turn(s)://username1:password1@host1:port1">

Flags : Read / Write


video-caps

“video-caps” GstCaps *

Governs what video codecs will be proposed

Flags : Read / Write

Default value :

video/x-vp8; video/x-h264; video/x-vp9; video/x-h265

Named constants

Web-rtcsink-congestion-control

Members

disabled (0) – Disabled: no congestion control is applied
homegrown (1) – Homegrown: simple sender-side heuristic
gcc (2) – Google Congestion Control algorithm

The results of the search are