GStreamer
open source multimedia framework
Home
Features
News
Annual Conference
Planet (Blogs)
Download
Applications
Security Center
GitLab
Developers
Documentation
Forum
File a Bug
Artwork
@gstreamer on Twitter
@gstreamer on Mastodon
#gstreamer on Matrix

GStreamer Projects: OpenGL ideas

The following ideas aim to improve GStreamer OpenGL library and plugins

Interested Mentors

Project idea 1: OpenGL audio visualizer

Port libvisual_gl gstgl elements from 0.10 to gst-1.0. Make it work with GLES2. This would enable an efficient gstreamer audio visualizer on embedded platforms. This project is also open to use another opensource audio visualizer API that support GL.

More details here

Project idea 2: VDPAU interoperability with GstGL

Use "GL_NV_vdpau_interop" OpenGL extension to make vdpauvideodec ! glimagesink avoid downloading/uploading the frame in between. Gstreamer vdpau elements are already working. We need to make them compatible with GstGL. This will enable a zero-copy path from the CPU point of view.

More details here

Project idea 3: Add suport for GstVideoOverlayCompositionMeta and apply it to new glcontrol and subtitle element

glcontrol: draws controls (play/pause button, seek bar, volume, quality) into a separate gltexture and attach it to the input buffer's GstVideoOverlayCompositionMeta. gltextoverlay: use an external GL Font API to build a separate gltexture and attach it to the input buffer's GstVideoOverlayCompositionMeta. Then a downstream element could call gst_video_overlay_composition_blend to show up the controls and subtitles.

gst-launch-1.0 playbin uri=foo video-sink="glcontrol ! glimagesink"

gst-launch-1.0 filesrc location=text.srt ! subparse ! text/x-raw ! gltextoverlay.text_sink ! glimagesink

More details for glcontrol here

More details for gltextoverlay here

Project idea 4: make other sinks compatible with gstgl

gstgl sink: glimagesink non-gstgl: Qt (qtglvideosink), Wayland (waylandsink), clutter (cluttersink/coglsink), cairo (cairosink).

Useful to benefit GstGL capabilities in these other video sinks. (Ex: zero-copy with OpenMAX decoder) The GstGL API is ready but some corner cases may be encountered. The student will need to improve API according.

gst-launch-1.0 src ! gleffects ! qtglvideosink

gst-launch-1.0 src ! omxvideodec ! qtglvideosink

More details here

Project idea 5: add supports for glCompressedTexImage2D

More details here

Project idea 6: new KTX texture parser

gst-launch-1.0 filesrc location=texture.ktx ! glktxparse ! glimagesink

KTX format

More details here

Project idea 7: make gst-gl compatible with DirectX

gl element should be able to handle dx surface as input.

gst-launch-1.0 src ! dxvadec ! glimagesink

gst-launch-1.0 videotestsrc ! gleffects ! d3dvideosink

Also investigate about accelerated video decoding on Windows (DXVA).

More details here

Student Notes:

Related links:


Report a problem on this page.