compositor

Compositor can accept AYUV, VUYA, ARGB and BGRA video streams. For each of the requested sink pads it will compare the incoming geometry and framerate to define the output parameters. Indeed output video frames will have the geometry of the biggest incoming video stream and the framerate of the fastest incoming one.

Compositor will do colorspace conversion.

Individual parameters for each input stream can be configured on the GstCompositorPad:

  • "xpos": The x-coordinate position of the top-left corner of the picture (#gint)
  • "ypos": The y-coordinate position of the top-left corner of the picture (#gint)
  • "width": The width of the picture; the input will be scaled if necessary (#gint)
  • "height": The height of the picture; the input will be scaled if necessary (#gint)
  • "alpha": The transparency of the picture; between 0.0 and 1.0. The blending is a simple copy when fully-transparent (0.0) and fully-opaque (1.0). (#gdouble)
  • "zorder": The z-order position of the picture in the composition (#guint)

Sample pipelines

 gst-launch-1.0 \
   videotestsrc pattern=1 ! \
   video/x-raw,format=AYUV,framerate=\(fraction\)10/1,width=100,height=100 ! \
   videobox border-alpha=0 top=-70 bottom=-70 right=-220 ! \
   compositor name=comp sink_0::alpha=0.7 sink_1::alpha=0.5 ! \
   videoconvert ! xvimagesink \
   videotestsrc ! \
   video/x-raw,format=AYUV,framerate=\(fraction\)5/1,width=320,height=240 ! comp.

A pipeline to demonstrate compositor used together with videobox. This should show a 320x240 pixels video test source with some transparency showing the background checker pattern. Another video test source with just the snow pattern of 100x100 pixels is overlaid on top of the first one on the left vertically centered with a small transparency showing the first video test source behind and the checker pattern under it. Note that the framerate of the output video is 10 frames per second.

 gst-launch-1.0 videotestsrc pattern=1 ! \
   video/x-raw, framerate=\(fraction\)10/1, width=100, height=100 ! \
   compositor name=comp ! videoconvert ! ximagesink \
   videotestsrc !  \
   video/x-raw, framerate=\(fraction\)5/1, width=320, height=240 ! comp.

A pipeline to demonstrate bgra comping. (This does not demonstrate alpha blending).

 gst-launch-1.0 videotestsrc pattern=1 ! \
   video/x-raw,format =I420, framerate=\(fraction\)10/1, width=100, height=100 ! \
   compositor name=comp ! videoconvert ! ximagesink \
   videotestsrc ! \
   video/x-raw,format=I420, framerate=\(fraction\)5/1, width=320, height=240 ! comp.

A pipeline to test I420

 gst-launch-1.0 compositor name=comp sink_1::alpha=0.5 sink_1::xpos=50 sink_1::ypos=50 ! \
   videoconvert ! ximagesink \
   videotestsrc pattern=snow timestamp-offset=3000000000 ! \
   "video/x-raw,format=AYUV,width=640,height=480,framerate=(fraction)30/1" ! \
   timeoverlay ! queue2 ! comp. \
   videotestsrc pattern=smpte ! \
   "video/x-raw,format=AYUV,width=800,height=600,framerate=(fraction)10/1" ! \
   timeoverlay ! queue2 ! comp.

A pipeline to demonstrate synchronized compositing (the second stream starts after 3 seconds)

Hierarchy

GObject
    ╰──GInitiallyUnowned
        ╰──GstObject
            ╰──GstElement
                ╰──GstAggregator
                    ╰──GstVideoAggregator
                        ╰──compositor

Factory details

Authors: – Wim Taymans , Sebastian Dröge

Classification:Filter/Editor/Video/Compositor

Rank – primary + 1

Plugin – gstcompositor

Package – GStreamer Base Plug-ins

Pad Templates

sink_%u

video/x-raw:
         format: { AYUV, VUYA, BGRA, ARGB, RGBA, ABGR, Y444, Y42B, YUY2, UYVY, YVYU, I420, YV12, NV12, NV21, Y41B, RGB, BGR, xRGB, xBGR, RGBx, BGRx }
          width: [ 1, 2147483647 ]
         height: [ 1, 2147483647 ]
      framerate: [ 0/1, 2147483647/1 ]

Presencerequest

Directionsink

Object typeGstCompositorPad


src

video/x-raw:
         format: { AYUV, VUYA, BGRA, ARGB, RGBA, ABGR, Y444, Y42B, YUY2, UYVY, YVYU, I420, YV12, NV12, NV21, Y41B, RGB, BGR, xRGB, xBGR, RGBx, BGRx }
          width: [ 1, 2147483647 ]
         height: [ 1, 2147483647 ]
      framerate: [ 0/1, 2147483647/1 ]

Presencealways

Directionsrc

Object typeGstAggregatorPad


Properties

background

“background” GstCompositorBackground *

Background type

Flags : Read / Write

Default value : checker (0)


latency

“latency” guint64

Additional latency in live mode to allow upstream to take longer to produce buffers for the current position (in nanoseconds)

Flags : Read / Write

Default value : 0


min-upstream-latency

“min-upstream-latency” guint64

When sources with a higher latency are expected to be plugged in dynamically after the aggregator has started playing, this allows overriding the minimum latency reported by the initial source(s). This is only taken into account when larger than the actually reported minimum latency. (nanoseconds)

Flags : Read / Write

Default value : 0


start-time

“start-time” guint64

Start time to use if start-time-selection=set

Flags : Read / Write

Default value : 18446744073709551615


start-time-selection

“start-time-selection” GstAggregatorStartTimeSelection *

Decides which start time is output

Flags : Read / Write

Default value : zero (0)


Named constants

GstCompositorBackground

The different backgrounds compositor can blend over.

Members

checker (0) – Checker pattern
black (1) – Black
white (2) – White
transparent (3) – Transparent Background to enable further compositing

GstCompositorOperator

The different blending operators that can be used by compositor.

See https://www.cairographics.org/operators/ for some explanation and visualizations.

Members

source (0) – Source
over (1) – Over
add (2) – Add

GstCompositorPad

GObject
    ╰──GInitiallyUnowned
        ╰──GstObject
            ╰──GstPad
                ╰──GstAggregatorPad
                    ╰──GstVideoAggregatorPad
                        ╰──GstVideoAggregatorConvertPad
                            ╰──GstCompositorPad

The opaque GstCompositorPad structure.


Signals

buffer-consumed

buffer_consumed_callback (GstElement * param_0,
                          GstBuffer * arg0,
                          gpointer udata)
def buffer_consumed_callback (param_0, arg0, udata):
    #python callback for the 'buffer-consumed' signal
function buffer_consumed_callback(param_0: GstElement * param_0, arg0: GstBuffer * arg0, udata: gpointer udata): {
    // javascript callback for the 'buffer-consumed' signal
}

Parameters:

param_0
No description available
arg0
No description available
udata
No description available

Flags: Run First


Properties

alpha

“alpha” gdouble

Alpha of the picture

Flags : Read / Write

Default value : 1


converter-config

“converter-config” GstStructure *

A GstStructure describing the configuration that should be used when scaling and converting this pad's video frames

Flags : Read / Write


emit-signals

“emit-signals” gboolean

Send signals to signal data consumption

Flags : Read / Write

Default value : false


height

“height” gint

Height of the picture

Flags : Read / Write

Default value : 0


max-last-buffer-repeat

“max-last-buffer-repeat” guint64

Repeat last buffer for time (in ns, -1=until EOS), behaviour on EOS is not affected

Flags : Read / Write

Default value : 18446744073709551615


operator

“operator” GstCompositorOperator *

Blending operator to use for blending this pad over the previous ones

Flags : Read / Write

Default value : over (1)


repeat-after-eos

“repeat-after-eos” gboolean

Repeat the last frame after EOS until all pads are EOS

Flags : Read / Write

Default value : false


width

“width” gint

Width of the picture

Flags : Read / Write

Default value : 0


xpos

“xpos” gint

X Position of the picture

Flags : Read / Write

Default value : 0


ypos

“ypos” gint

Y Position of the picture

Flags : Read / Write

Default value : 0


zorder

“zorder” guint

Z Order of the picture

Flags : Read / Write

Default value : 0


The results of the search are