rtpst2022-1-fecdec

This element takes as input a media stream and up to two FEC streams as described in SMPTE 2022-1: Forward Error Correction for Real-Time Video/Audio Transport Over IP Networks, and makes use of the FEC packets to recover media packets that may have gotten lost.

Design

The approach picked for this element is to proactively reconstruct missing packets as soon as possible. When a FEC packet arrives, the element immediately checks whether a media packet in the row / column it protects can be reconstructed.

Similarly, when a media packet comes in, the element checks whether it has already received a corresponding packet in both the column and row the packet belongs to, and if so goes through the first step listed above.

This process is repeated recursively, allowing for recoveries over one dimension to unblock recoveries over the other.

In perfect networking conditions, this incurs next to no overhead as FEC packets will arrive after the media packets, causing no reconstruction to take place, just a few checks upon chaining.

sender / receiver example

gst-launch-1.0 \
  rtpbin name=rtp fec-encoders='fec,0="rtpst2022-1-fecenc\ rows\=5\ columns\=5";' \
  uridecodebin uri=file:///path/to/video/file ! x264enc key-int-max=60 tune=zerolatency ! \
    queue ! mpegtsmux ! rtpmp2tpay ssrc=0 ! rtp.send_rtp_sink_0 \
  rtp.send_rtp_src_0 ! udpsink host=127.0.0.1 port=5000 \
  rtp.send_fec_src_0_0 ! udpsink host=127.0.0.1 port=5002 async=false \
  rtp.send_fec_src_0_1 ! udpsink host=127.0.0.1 port=5004 async=false

gst-launch-1.0 \
  rtpbin latency=500 fec-decoders='fec,0="rtpst2022-1-fecdec\ size-time\=1000000000";' name=rtp \
  udpsrc address=127.0.0.1 port=5002 caps="application/x-rtp, payload=96" ! queue ! rtp.recv_fec_sink_0_0 \
  udpsrc address=127.0.0.1 port=5004 caps="application/x-rtp, payload=96" ! queue ! rtp.recv_fec_sink_0_1 \
  udpsrc address=127.0.0.1 port=5000 caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=mp2t, payload=33" ! \
    queue ! netsim drop-probability=0.05 ! rtp.recv_rtp_sink_0 \
  rtp. ! decodebin ! videoconvert ! queue ! autovideosink

With the above command line, as the media packet size is constant, the fec overhead can be approximated to the number of fec packets per 2-d matrix of media packet, here 10 fec packets for each 25 media packets.

Increasing the number of rows and columns will decrease the overhead, but obviously increase the likelihood of recovery failure for lost packets on the receiver side.