RTP in Gstreamer 0.10
The RTP stack described below on this page is in place and works, but a new and improved one is in development and you can find the latest details on that on the Farsight Wiki page.
Philippe Khalaf <philippe.kalaf at collabora.co.uk>
Any questions you have you can find me on #farsight or #gstreamer on freenode, my nickname is Burger.
This document describes how to get RTP working in GStreamer 0.10.
The elements are currently being developed in Farsight's darcs repository. The
required module is gst-plugins-farsight.
The elements should also be available in GStreamer Git and future releases
(releases after May 2006), but latest edge will be on the farsight darcs.
There are 3 main elements for RTP :
jrtplib and jthread are optional dependencies for rtpbin. It is required if you wish to use more advanced RTP features such as
RTCP, SSRC control and multi-user (multiple destinations/sources). If they are not installed rtpbin will compile without them.
Current state : (17/04/06)
- Optional jrtplib dependency
- rtpjitterbuffer added to rtpbin
- bypass-udp property added that allows to connect to rtpbin for transfering
over other sink/src that udp (i.e. icesrc/sink for libjingle)
- sockfd properties added to allow to set an external sockfd for udp
- pt-map property added to rtpbin that defines supported payloads and their
- queue-delay property added to rtpbin for the jitter buffer size
(jitterbuffer removed from basedepayloader class)
- many other small improvements
(15/08/05) Some more changes to RTPBin :
- No more GstRTPBuffer, instead we use normal GstBuffer's with full rtp
packets inside of them. gstrtpbuffer.[ch] contain all the helper parse
functions that allow to build and read/write the rtp headers.
- No more modes, the mode is determined by on request pads. So depending on
your requested pads a mode will be selected.
- Parameter to turn on/off RTCP
(20/07/05) Brand spanking new RTPBin element. It's dynamic. It has 6 modes :
- Send and Receive
- Send Only
- Receive Only
- Send and Receive without RTCP
- Send Only without RTCP
- Receive Only without RTCP
I think they are self-explanatory.
(14/07/05) Basic RTP support is in the CVS. By basic I mean this is the first version of RTP, at it's most simple form. It has RTCP support, meaning it will receive and send RTCP packets, and process them. That information is still not being used actively by the system.
It also provides a depayloader base class. This shall be used by codec specific depayloaders. It implements generic functions such as buffering, reordering and dropping packets.
How to use it
Here is what you get :
- First you need to have gstreamer, gst-base-plugins and gst-plugins. I use
CVS but most 0.10 releases should do fine as well
- Then you need to compile and install gst-plugins-farsight. If you want
jrtplib support you need to install jrtplib and jthread before installing
gst-plugins-farsight. You need to get jrtplib and jthread from here
rtpbin : This is the bin that does it all. It adapts dynamically to your needs based on the requested pads. It contains the rtpjitterbuffer for extra
jitter control. GStreamer's sound sources already contains a jitter buffer, so
this is only usefull for packet reodering. The buffer can be disabled by setting
queue-delay to 0.
jrtpjitterbuffer : This element is an RTP buffer that controls network jitter and reorders packets. It also dumps packets that arrive to late. The
jitterbuffer still has some issues. For example it uses a thread/timer to
release packets from the queue. Ideally the sink would "pull" the buffers out of
it instead. It is not heavily tested and might have some leaks.
rtpdemux : This element usually sits on the rtpbin src pad and will detect any new payload types that arrive in the RTP stream. It will then create a pad for that new payload and you can connect a depayloader/decoder pipeline to that pad.
BaseRTPDepayload : A base depayloader class that should be used by any RTP depayloaders you write (h263depayloader is an example depayloader element that uses BaseRTPDepayload class).
BaseRTPPayload : A base payloader class that should be used by any RTP payloaders you write.
BaseRTPAudioPayload Coming soon
(17/04/06) : A base payloader class for audio codecs. It supports
frame based and sample based audio codecs (constant bitrate).
If you want to use rtpbin, there are a few things you need to know :
Image 1 - Internals of RTPBin with JRTPLIB support
- To use it you need to set the localport, otherwise it won't work.
- If sending data, the destinations must be added with the "destinations" property or the "add-destination" signal. This can contain one or more ip:port combinations with the following syntax (127.0.0.1:5000;192.168.1.101:4000;).
- Any buffers that go into sink MUST be rtp buffers.
src also gives out full rtp buffers that contain important information for the depayloader.
Image 2 - Internals of RTPBin without JRTPLIB support
- Inside the bin, rtpsink/rtcpsink MUST receive GstNetBuffers.
This contains important source information needed if using jrtplib . Also, rtpsrc/rtcpsrc sends out GstNetBuffers that the sinks must use to send to the given destinations in the GstNetBuffers. (dynudpsink does this as an example).
- If you need to reroute the packet flow, you will need to rewrite another rtpbin.
Here is what a pipeline would normally look like with the different modes:
Here is one using the rtpdemuxer element :
- As a note for the first part, its easier if the payloader is embeded in the encoder because it requires some encoding specific marks and data, but it is possible to have it separate.
- Also I have no tested/verified RTSP compatiblity yet, but it has been taken into account during the design. There are a few RTSP elements written by wtay that can probably be used successfully here.