GStreamer
open source multimedia framework
Home
Features
News
Annual Conference
Planet (Blogs)
Download
Applications
Developers
Documentation
Mailing Lists
File a Bug
Bug Lists
Artwork
Follow @gstreamer on Twitter

RTP in Gstreamer 0.10

The RTP stack described below on this page is in place and works, but a new and improved one is in development and you can find the latest details on that on the Farsight Wiki page.

Philippe Khalaf <philippe.kalaf at collabora.co.uk>

Any questions you have you can find me on #farsight or #gstreamer on freenode, my nickname is Burger.

Intro

This document describes how to get RTP working in GStreamer 0.10.

The elements are currently being developed in Farsight's darcs repository. The required module is gst-plugins-farsight.

The elements should also be available in GStreamer Git and future releases (releases after May 2006), but latest edge will be on the farsight darcs.

There are 3 main elements for RTP :

  • rtpbin
  • rtpjitterbuffer
  • rtpdemux.

jrtplib and jthread are optional dependencies for rtpbin. It is required if you wish to use more advanced RTP features such as RTCP, SSRC control and multi-user (multiple destinations/sources). If they are not installed rtpbin will compile without them.

Current state : (17/04/06)

  • Optional jrtplib dependency
  • rtpjitterbuffer added to rtpbin
  • bypass-udp property added that allows to connect to rtpbin for transfering over other sink/src that udp (i.e. icesrc/sink for libjingle)
  • sockfd properties added to allow to set an external sockfd for udp
  • pt-map property added to rtpbin that defines supported payloads and their clock rates
  • queue-delay property added to rtpbin for the jitter buffer size (jitterbuffer removed from basedepayloader class)
  • many other small improvements

(15/08/05) Some more changes to RTPBin :

  • No more GstRTPBuffer, instead we use normal GstBuffer's with full rtp packets inside of them. gstrtpbuffer.[ch] contain all the helper parse functions that allow to build and read/write the rtp headers.
  • No more modes, the mode is determined by on request pads. So depending on your requested pads a mode will be selected.
  • Parameter to turn on/off RTCP

(20/07/05) Brand spanking new RTPBin element. It's dynamic. It has 6 modes :

  • Send and Receive
  • Send Only
  • Receive Only
  • Send and Receive without RTCP
  • Send Only without RTCP
  • Receive Only without RTCP

I think they are self-explanatory.

(14/07/05) Basic RTP support is in the CVS. By basic I mean this is the first version of RTP, at it's most simple form. It has RTCP support, meaning it will receive and send RTCP packets, and process them. That information is still not being used actively by the system. It also provides a depayloader base class. This shall be used by codec specific depayloaders. It implements generic functions such as buffering, reordering and dropping packets.

How to use it

  • First you need to have gstreamer, gst-base-plugins and gst-plugins. I use CVS but most 0.10 releases should do fine as well
  • Then you need to compile and install gst-plugins-farsight. If you want jrtplib support you need to install jrtplib and jthread before installing gst-plugins-farsight. You need to get jrtplib and jthread from here for now.
Here is what you get :
  • rtpbin : This is the bin that does it all. It adapts dynamically to your needs based on the requested pads. It contains the rtpjitterbuffer for extra jitter control. GStreamer's sound sources already contains a jitter buffer, so this is only usefull for packet reodering. The buffer can be disabled by setting queue-delay to 0.
  • jrtpjitterbuffer : This element is an RTP buffer that controls network jitter and reorders packets. It also dumps packets that arrive to late. The jitterbuffer still has some issues. For example it uses a thread/timer to release packets from the queue. Ideally the sink would "pull" the buffers out of it instead. It is not heavily tested and might have some leaks.
  • rtpdemux : This element usually sits on the rtpbin src pad and will detect any new payload types that arrive in the RTP stream. It will then create a pad for that new payload and you can connect a depayloader/decoder pipeline to that pad.
  • BaseRTPDepayload : A base depayloader class that should be used by any RTP depayloaders you write (h263depayloader is an example depayloader element that uses BaseRTPDepayload class).
  • BaseRTPPayload : A base payloader class that should be used by any RTP payloaders you write.
  • BaseRTPAudioPayload Coming soon (17/04/06) : A base payloader class for audio codecs. It supports frame based and sample based audio codecs (constant bitrate).

If you want to use rtpbin, there are a few things you need to know :

  • To use it you need to set the localport, otherwise it won't work.
  • If sending data, the destinations must be added with the "destinations" property or the "add-destination" signal. This can contain one or more ip:port combinations with the following syntax (127.0.0.1:5000;192.168.1.101:4000;).
  • Any buffers that go into sink MUST be rtp buffers.
  • src also gives out full rtp buffers that contain important information for the depayloader.
Image 1 - Internals of RTPBin with JRTPLIB support
Image 2 - Internals of RTPBin without JRTPLIB support

Notes
  • Inside the bin, rtpsink/rtcpsink MUST receive GstNetBuffers. This contains important source information needed if using jrtplib . Also, rtpsrc/rtcpsrc sends out GstNetBuffers that the sinks must use to send to the given destinations in the GstNetBuffers. (dynudpsink does this as an example).
  • If you need to reroute the packet flow, you will need to rewrite another rtpbin.

Pipeline construction

Here is what a pipeline would normally look like with the different modes:

Here is one using the rtpdemuxer element :

Notes

  • As a note for the first part, its easier if the payloader is embeded in the encoder because it requires some encoding specific marks and data, but it is possible to have it separate.
  • Also I have no tested/verified RTSP compatiblity yet, but it has been taken into account during the design. There are a few RTSP elements written by wtay that can probably be used successfully here.


Report a problem on this page.