gstvideooverlaycomposition

gstvideooverlaycomposition — Video Buffer Overlay Compositions (Subtitles, Logos)

Functions

GstVideoOverlayComposition * gst_video_overlay_composition_new ()
GstVideoOverlayComposition * gst_video_overlay_composition_ref ()
void gst_video_overlay_composition_unref ()
void gst_video_overlay_composition_add_rectangle ()
guint gst_video_overlay_composition_n_rectangles ()
GstVideoOverlayRectangle * gst_video_overlay_composition_get_rectangle ()
guint gst_video_overlay_composition_get_seqnum ()
GstVideoOverlayComposition * gst_video_overlay_composition_copy ()
GstVideoOverlayComposition * gst_video_overlay_composition_make_writable ()
gboolean gst_video_overlay_composition_blend ()
const GstMetaInfo * gst_video_overlay_composition_meta_get_info ()
GstVideoOverlayCompositionMeta * gst_buffer_add_video_overlay_composition_meta ()
#define gst_buffer_get_video_overlay_composition_meta()
#define gst_buffer_remove_video_overlay_composition_meta()
GstVideoOverlayRectangle * gst_video_overlay_rectangle_new_raw ()
GstVideoOverlayRectangle * gst_video_overlay_rectangle_ref ()
void gst_video_overlay_rectangle_unref ()
GstBuffer * gst_video_overlay_rectangle_get_pixels_raw ()
GstBuffer * gst_video_overlay_rectangle_get_pixels_argb ()
GstBuffer * gst_video_overlay_rectangle_get_pixels_ayuv ()
GstBuffer * gst_video_overlay_rectangle_get_pixels_unscaled_raw ()
GstBuffer * gst_video_overlay_rectangle_get_pixels_unscaled_argb ()
GstBuffer * gst_video_overlay_rectangle_get_pixels_unscaled_ayuv ()
gboolean gst_video_overlay_rectangle_get_render_rectangle ()
guint gst_video_overlay_rectangle_get_seqnum ()
void gst_video_overlay_rectangle_set_render_rectangle ()
GstVideoOverlayRectangle * gst_video_overlay_rectangle_copy ()
GstVideoOverlayFormatFlags gst_video_overlay_rectangle_get_flags ()
gfloat gst_video_overlay_rectangle_get_global_alpha ()
void gst_video_overlay_rectangle_set_global_alpha ()

Types and Values

Object Hierarchy


Includes

#include <gst/video/video-overlay-composition.h>

Description

Functions to create and handle overlay compositions on video buffers.

An overlay composition describes one or more overlay rectangles to be blended on top of a video buffer.

This API serves two main purposes:

GstOverlayComposition
  • it can be used to attach overlay information (subtitles or logos) to non-raw video buffers such as GL/VAAPI/VDPAU surfaces. The actual blending of the overlay can then be done by e.g. the video sink that processes these non-raw buffers.
  • it can also be used to blend overlay rectangles on top of raw video buffers, thus consolidating blending functionality for raw video in one place.

Functions

gst_video_overlay_composition_new ()

GstVideoOverlayComposition *
gst_video_overlay_composition_new (GstVideoOverlayRectangle *rectangle);

Creates a new video overlay composition object to hold one or more overlay rectangles.

Parameters

rectangle

a GstVideoOverlayRectangle to add to the composition.

[transfer none]

Returns

a new GstVideoOverlayComposition. Unref with gst_video_overlay_composition_unref() when no longer needed.

[transfer full]


gst_video_overlay_composition_ref ()

GstVideoOverlayComposition *
gst_video_overlay_composition_ref (GstVideoOverlayComposition *comp);

Increases the refcount of the given composition by one.

Note that the refcount affects the writeability of comp , use gst_video_overlay_composition_make_writable() to ensure a composition and its rectangles can be modified.

Parameters

Returns

comp .

[transfer full]


gst_video_overlay_composition_unref ()

void
gst_video_overlay_composition_unref (GstVideoOverlayComposition *comp);

Decreases the refcount of the composition. If the refcount reaches 0, the composition will be freed.

Parameters

comp

a GstVideoOverlayComposition.

[transfer full]

gst_video_overlay_composition_add_rectangle ()

void
gst_video_overlay_composition_add_rectangle
                               (GstVideoOverlayComposition *comp,
                                GstVideoOverlayRectangle *rectangle);

Adds an overlay rectangle to an existing overlay composition object. This must be done right after creating the overlay composition.

Parameters

comp

a GstVideoOverlayComposition

 

rectangle

a GstVideoOverlayRectangle to add to the composition.

[transfer none]

gst_video_overlay_composition_n_rectangles ()

guint
gst_video_overlay_composition_n_rectangles
                               (GstVideoOverlayComposition *comp);

Returns the number of GstVideoOverlayRectangles contained in comp .

Parameters

Returns

the number of rectangles


gst_video_overlay_composition_get_rectangle ()

GstVideoOverlayRectangle *
gst_video_overlay_composition_get_rectangle
                               (GstVideoOverlayComposition *comp,
                                guint n);

Returns the n -th GstVideoOverlayRectangle contained in comp .

Parameters

comp

a GstVideoOverlayComposition

 

n

number of the rectangle to get

 

Returns

the n -th rectangle, or NULL if n is out of bounds. Will not return a new reference, the caller will need to obtain her own reference using gst_video_overlay_rectangle_ref() if needed.

[transfer none]


gst_video_overlay_composition_get_seqnum ()

guint
gst_video_overlay_composition_get_seqnum
                               (GstVideoOverlayComposition *comp);

Returns the sequence number of this composition. Sequence numbers are monotonically increasing and unique for overlay compositions and rectangles (meaning there will never be a rectangle with the same sequence number as a composition).

Parameters

Returns

the sequence number of comp


gst_video_overlay_composition_copy ()

GstVideoOverlayComposition *
gst_video_overlay_composition_copy (GstVideoOverlayComposition *comp);

Makes a copy of comp and all contained rectangles, so that it is possible to modify the composition and contained rectangles (e.g. add additional rectangles or change the render co-ordinates or render dimension). The actual overlay pixel data buffers contained in the rectangles are not copied.

Parameters

comp

a GstVideoOverlayComposition to copy.

[transfer none]

Returns

a new GstVideoOverlayComposition equivalent to comp .

[transfer full]


gst_video_overlay_composition_make_writable ()

GstVideoOverlayComposition *
gst_video_overlay_composition_make_writable
                               (GstVideoOverlayComposition *comp);

Takes ownership of comp and returns a version of comp that is writable (i.e. can be modified). Will either return comp right away, or create a new writable copy of comp and unref comp itself. All the contained rectangles will also be copied, but the actual overlay pixel data buffers contained in the rectangles are not copied.

Parameters

comp

a GstVideoOverlayComposition to copy.

[transfer full]

Returns

a writable GstVideoOverlayComposition equivalent to comp .

[transfer full]


gst_video_overlay_composition_blend ()

gboolean
gst_video_overlay_composition_blend (GstVideoOverlayComposition *comp,
                                     GstVideoFrame *video_buf);

Blends the overlay rectangles in comp on top of the raw video data contained in video_buf . The data in video_buf must be writable and mapped appropriately.

Parameters

comp

a GstVideoOverlayComposition

 

video_buf

a GstVideoFrame containing raw video data in a supported format

 

gst_video_overlay_composition_meta_get_info ()

const GstMetaInfo *
gst_video_overlay_composition_meta_get_info
                               (void);

gst_buffer_add_video_overlay_composition_meta ()

GstVideoOverlayCompositionMeta *
gst_buffer_add_video_overlay_composition_meta
                               (GstBuffer *buf,
                                GstVideoOverlayComposition *comp);

Sets an overlay composition on a buffer. The buffer will obtain its own reference to the composition, meaning this function does not take ownership of comp .

Parameters

buf

a GstBuffer

 

comp

a GstVideoOverlayComposition.

[allow-none]

gst_buffer_get_video_overlay_composition_meta()

#define             gst_buffer_get_video_overlay_composition_meta(b)

gst_buffer_remove_video_overlay_composition_meta()

#define             gst_buffer_remove_video_overlay_composition_meta(b,m)

gst_video_overlay_rectangle_new_raw ()

GstVideoOverlayRectangle *
gst_video_overlay_rectangle_new_raw (GstBuffer *pixels,
                                     gint render_x,
                                     gint render_y,
                                     guint render_width,
                                     guint render_height,
                                     GstVideoOverlayFormatFlags flags);

Creates a new video overlay rectangle with ARGB or AYUV pixel data. The layout in case of ARGB of the components in memory is B-G-R-A on little-endian platforms (corresponding to GST_VIDEO_FORMAT_BGRA) and A-R-G-B on big-endian platforms (corresponding to GST_VIDEO_FORMAT_ARGB). In other words, pixels are treated as 32-bit words and the lowest 8 bits then contain the blue component value and the highest 8 bits contain the alpha component value. Unless specified in the flags, the RGB values are non-premultiplied. This is the format that is used by most hardware, and also many rendering libraries such as Cairo, for example. The pixel data buffer must have GstVideoMeta set.

Parameters

pixels

a GstBuffer pointing to the pixel memory.

[transfer none]

render_x

the X co-ordinate on the video where the top-left corner of this overlay rectangle should be rendered to

 

render_y

the Y co-ordinate on the video where the top-left corner of this overlay rectangle should be rendered to

 

render_width

the render width of this rectangle on the video

 

render_height

the render height of this rectangle on the video

 

flags

flags

 

Returns

a new GstVideoOverlayRectangle. Unref with gst_video_overlay_rectangle_unref() when no longer needed.

[transfer full]


gst_video_overlay_rectangle_ref ()

GstVideoOverlayRectangle *
gst_video_overlay_rectangle_ref (GstVideoOverlayRectangle *comp);

Increases the refcount of the given rectangle by one.

Note that the refcount affects the writeability of comp , use gst_video_overlay_rectangle_copy() to ensure a rectangle can be modified (there is no gst_video_overlay_rectangle_make_writable() because it is unlikely that someone will hold the single reference to the rectangle and not know that that's the case).

Parameters

Returns

comp .

[transfer full]


gst_video_overlay_rectangle_unref ()

void
gst_video_overlay_rectangle_unref (GstVideoOverlayRectangle *comp);

Decreases the refcount of the rectangle. If the refcount reaches 0, the rectangle will be freed.

Parameters

comp

a GstVideoOverlayRectangle.

[transfer full]

gst_video_overlay_rectangle_get_pixels_raw ()

GstBuffer *
gst_video_overlay_rectangle_get_pixels_raw
                               (GstVideoOverlayRectangle *rectangle,
                                GstVideoOverlayFormatFlags flags);

Parameters

rectangle

a GstVideoOverlayRectangle

 

flags

flags If a global_alpha value != 1 is set for the rectangle, the caller should set the GST_VIDEO_OVERLAY_FORMAT_FLAG_GLOBAL_ALPHA flag if he wants to apply global-alpha himself. If the flag is not set global_alpha is applied internally before returning the pixel-data.

 

Returns

a GstBuffer holding the pixel data with format as originally provided and specified in video meta with width and height of the render dimensions as per gst_video_overlay_rectangle_get_render_rectangle(). This function does not return a reference, the caller should obtain a reference of her own with gst_buffer_ref() if needed.

[transfer none]


gst_video_overlay_rectangle_get_pixels_argb ()

GstBuffer *
gst_video_overlay_rectangle_get_pixels_argb
                               (GstVideoOverlayRectangle *rectangle,
                                GstVideoOverlayFormatFlags flags);

Parameters

rectangle

a GstVideoOverlayRectangle

 

flags

flags If a global_alpha value != 1 is set for the rectangle, the caller should set the GST_VIDEO_OVERLAY_FORMAT_FLAG_GLOBAL_ALPHA flag if he wants to apply global-alpha himself. If the flag is not set global_alpha is applied internally before returning the pixel-data.

 

Returns

a GstBuffer holding the ARGB pixel data with width and height of the render dimensions as per gst_video_overlay_rectangle_get_render_rectangle(). This function does not return a reference, the caller should obtain a reference of her own with gst_buffer_ref() if needed.

[transfer none]


gst_video_overlay_rectangle_get_pixels_ayuv ()

GstBuffer *
gst_video_overlay_rectangle_get_pixels_ayuv
                               (GstVideoOverlayRectangle *rectangle,
                                GstVideoOverlayFormatFlags flags);

Parameters

rectangle

a GstVideoOverlayRectangle

 

flags

flags If a global_alpha value != 1 is set for the rectangle, the caller should set the GST_VIDEO_OVERLAY_FORMAT_FLAG_GLOBAL_ALPHA flag if he wants to apply global-alpha himself. If the flag is not set global_alpha is applied internally before returning the pixel-data.

 

Returns

a GstBuffer holding the AYUV pixel data with width and height of the render dimensions as per gst_video_overlay_rectangle_get_render_rectangle(). This function does not return a reference, the caller should obtain a reference of her own with gst_buffer_ref() if needed.

[transfer none]


gst_video_overlay_rectangle_get_pixels_unscaled_raw ()

GstBuffer *
gst_video_overlay_rectangle_get_pixels_unscaled_raw
                               (GstVideoOverlayRectangle *rectangle,
                                GstVideoOverlayFormatFlags flags);

Retrieves the pixel data as it is. This is useful if the caller can do the scaling itself when handling the overlaying. The rectangle will need to be scaled to the render dimensions, which can be retrieved using gst_video_overlay_rectangle_get_render_rectangle().

Parameters

rectangle

a GstVideoOverlayRectangle

 

flags

flags. If a global_alpha value != 1 is set for the rectangle, the caller should set the GST_VIDEO_OVERLAY_FORMAT_FLAG_GLOBAL_ALPHA flag if he wants to apply global-alpha himself. If the flag is not set global_alpha is applied internally before returning the pixel-data.

 

Returns

a GstBuffer holding the pixel data with GstVideoMeta set. This function does not return a reference, the caller should obtain a reference of her own with gst_buffer_ref() if needed.

[transfer none]


gst_video_overlay_rectangle_get_pixels_unscaled_argb ()

GstBuffer *
gst_video_overlay_rectangle_get_pixels_unscaled_argb
                               (GstVideoOverlayRectangle *rectangle,
                                GstVideoOverlayFormatFlags flags);

Retrieves the pixel data as it is. This is useful if the caller can do the scaling itself when handling the overlaying. The rectangle will need to be scaled to the render dimensions, which can be retrieved using gst_video_overlay_rectangle_get_render_rectangle().

Parameters

rectangle

a GstVideoOverlayRectangle

 

flags

flags. If a global_alpha value != 1 is set for the rectangle, the caller should set the GST_VIDEO_OVERLAY_FORMAT_FLAG_GLOBAL_ALPHA flag if he wants to apply global-alpha himself. If the flag is not set global_alpha is applied internally before returning the pixel-data.

 

Returns

a GstBuffer holding the ARGB pixel data with GstVideoMeta set. This function does not return a reference, the caller should obtain a reference of her own with gst_buffer_ref() if needed.

[transfer none]


gst_video_overlay_rectangle_get_pixels_unscaled_ayuv ()

GstBuffer *
gst_video_overlay_rectangle_get_pixels_unscaled_ayuv
                               (GstVideoOverlayRectangle *rectangle,
                                GstVideoOverlayFormatFlags flags);

Retrieves the pixel data as it is. This is useful if the caller can do the scaling itself when handling the overlaying. The rectangle will need to be scaled to the render dimensions, which can be retrieved using gst_video_overlay_rectangle_get_render_rectangle().

Parameters

rectangle

a GstVideoOverlayRectangle

 

flags

flags. If a global_alpha value != 1 is set for the rectangle, the caller should set the GST_VIDEO_OVERLAY_FORMAT_FLAG_GLOBAL_ALPHA flag if he wants to apply global-alpha himself. If the flag is not set global_alpha is applied internally before returning the pixel-data.

 

Returns

a GstBuffer holding the AYUV pixel data with GstVideoMeta set. This function does not return a reference, the caller should obtain a reference of her own with gst_buffer_ref() if needed.

[transfer none]


gst_video_overlay_rectangle_get_render_rectangle ()

gboolean
gst_video_overlay_rectangle_get_render_rectangle
                               (GstVideoOverlayRectangle *rectangle,
                                gint *render_x,
                                gint *render_y,
                                guint *render_width,
                                guint *render_height);

Retrieves the render position and render dimension of the overlay rectangle on the video.

Parameters

rectangle

a GstVideoOverlayRectangle

 

render_x

address where to store the X render offset.

[out][allow-none]

render_y

address where to store the Y render offset.

[out][allow-none]

render_width

address where to store the render width.

[out][allow-none]

render_height

address where to store the render height.

[out][allow-none]

Returns

TRUE if valid render dimensions were retrieved.


gst_video_overlay_rectangle_get_seqnum ()

guint
gst_video_overlay_rectangle_get_seqnum
                               (GstVideoOverlayRectangle *rectangle);

Returns the sequence number of this rectangle. Sequence numbers are monotonically increasing and unique for overlay compositions and rectangles (meaning there will never be a rectangle with the same sequence number as a composition).

Using the sequence number of a rectangle as an indicator for changed pixel-data of a rectangle is dangereous. Some API calls, like e.g. gst_video_overlay_rectangle_set_global_alpha(), automatically update the per rectangle sequence number, which is misleading for renderers/ consumers, that handle global-alpha themselves. For them the pixel-data returned by gst_video_overlay_rectangle_get_pixels_*() wont be different for different global-alpha values. In this case a renderer could also use the GstBuffer pointers as a hint for changed pixel-data.

Parameters

rectangle

a GstVideoOverlayRectangle

 

Returns

the sequence number of rectangle


gst_video_overlay_rectangle_set_render_rectangle ()

void
gst_video_overlay_rectangle_set_render_rectangle
                               (GstVideoOverlayRectangle *rectangle,
                                gint render_x,
                                gint render_y,
                                guint render_width,
                                guint render_height);

Sets the render position and dimensions of the rectangle on the video. This function is mainly for elements that modify the size of the video in some way (e.g. through scaling or cropping) and need to adjust the details of any overlays to match the operation that changed the size.

rectangle must be writable, meaning its refcount must be 1. You can make the rectangles inside a GstVideoOverlayComposition writable using gst_video_overlay_composition_make_writable() or gst_video_overlay_composition_copy().

Parameters

rectangle

a GstVideoOverlayRectangle

 

render_x

render X position of rectangle on video

 

render_y

render Y position of rectangle on video

 

render_width

render width of rectangle

 

render_height

render height of rectangle

 

gst_video_overlay_rectangle_copy ()

GstVideoOverlayRectangle *
gst_video_overlay_rectangle_copy (GstVideoOverlayRectangle *rectangle);

Makes a copy of rectangle , so that it is possible to modify it (e.g. to change the render co-ordinates or render dimension). The actual overlay pixel data buffers contained in the rectangle are not copied.

Parameters

rectangle

a GstVideoOverlayRectangle to copy.

[transfer none]

Returns

a new GstVideoOverlayRectangle equivalent to rectangle .

[transfer full]


gst_video_overlay_rectangle_get_flags ()

GstVideoOverlayFormatFlags
gst_video_overlay_rectangle_get_flags (GstVideoOverlayRectangle *rectangle);

Retrieves the flags associated with a GstVideoOverlayRectangle. This is useful if the caller can handle both premultiplied alpha and non premultiplied alpha, for example. By knowing whether the rectangle uses premultiplied or not, it can request the pixel data in the format it is stored in, to avoid unnecessary conversion.

Parameters

rectangle

a GstVideoOverlayRectangle

 

Returns

the GstVideoOverlayFormatFlags associated with the rectangle.


gst_video_overlay_rectangle_get_global_alpha ()

gfloat
gst_video_overlay_rectangle_get_global_alpha
                               (GstVideoOverlayRectangle *rectangle);

Retrieves the global-alpha value associated with a GstVideoOverlayRectangle.

Parameters

rectangle

a GstVideoOverlayRectangle

 

Returns

the global-alpha value associated with the rectangle.


gst_video_overlay_rectangle_set_global_alpha ()

void
gst_video_overlay_rectangle_set_global_alpha
                               (GstVideoOverlayRectangle *rectangle,
                                gfloat global_alpha);

Sets the global alpha value associated with a GstVideoOverlayRectangle. Per- pixel alpha values are multiplied with this value. Valid values: 0 <= global_alpha <= 1; 1 to deactivate.

# rectangle must be writable, meaning its refcount must be 1. You can make the rectangles inside a GstVideoOverlayComposition writable using gst_video_overlay_composition_make_writable() or gst_video_overlay_composition_copy().

Parameters

rectangle

a GstVideoOverlayRectangle

 

Types and Values

GstVideoOverlayComposition

typedef struct _GstVideoOverlayComposition GstVideoOverlayComposition;

An opaque video overlay composition object. A composition contains multiple overlay rectangles.


GST_VIDEO_OVERLAY_COMPOSITION_BLEND_FORMATS

#define             GST_VIDEO_OVERLAY_COMPOSITION_BLEND_FORMATS

Video formats supported by gst_video_overlay_composition_blend(), for use in overlay elements' pad template caps.

Since 1.2


struct GstVideoOverlayCompositionMeta

struct GstVideoOverlayCompositionMeta {
  GstMeta meta;

  GstVideoOverlayComposition *overlay;
};

Extra buffer metadata describing image overlay data.

Members

GstMeta meta;

parent GstMeta

 

GstVideoOverlayComposition *overlay;

the attached GstVideoOverlayComposition

 

enum GstVideoOverlayFormatFlags

Overlay format flags.

Members

GST_VIDEO_OVERLAY_FORMAT_FLAG_NONE

no flags

 

GST_VIDEO_OVERLAY_FORMAT_FLAG_PREMULTIPLIED_ALPHA

RGB are premultiplied by A/255.

 

GST_VIDEO_OVERLAY_FORMAT_FLAG_GLOBAL_ALPHA

a global-alpha value != 1 is set.

 

GstVideoOverlayRectangle

typedef struct _GstVideoOverlayRectangle GstVideoOverlayRectangle;

An opaque video overlay rectangle object. A rectangle contains a single overlay rectangle which can be added to a composition.