RTP formats overview

Understand how gortsplib handles RTP payload formats for audio and video.

Every RTP stream carries a specific payload format that defines how media is encoded into packets. gortsplib models each format as a Go struct that implements the Format interface, giving you typed access to codec parameters and built-in RTP packetizers/depacketizers.

The Format interface

All format structs live in the github.com/bluenviron/gortsplib/v5/pkg/format package and satisfy this interface:

type Format interface {
    // PayloadType returns the RTP payload type number (0-127).
    PayloadType() uint8

    // ClockRate returns the RTP clock rate in Hz.
    ClockRate() int

    // RTPMap returns the rtpmap SDP attribute value, e.g. "H264/90000".
    RTPMap() string

    // FMTP returns the fmtp SDP attribute as a key/value map.
    FMTP() map[string]string

    // PTSEqualsDTS reports whether PTS equals DTS for a given RTP packet.
    PTSEqualsDTS(*rtp.Packet) bool

    // Codec returns a human-readable codec name.
    // Deprecated: will be removed in the next version.
    Codec() string
}

Most concrete format types additionally expose CreateEncoder() and CreateDecoder() methods that return format-specific RTP encoder/decoder objects.

Method	Purpose
PayloadType()	Identifies the payload within an RTP stream
ClockRate()	Used to convert RTP timestamps to wall-clock time
RTPMap()	Written into SDP a=rtpmap lines
FMTP()	Written into SDP a=fmtp lines
CreateEncoder()	Returns a packetizer that turns codec data into RTP packets
CreateDecoder()	Returns a depacketizer that reassembles RTP packets into codec data

Formats in a session description

When you call client.Describe(), the library parses the SDP and returns a *description.Session. Each description.Media in desc.Medias holds a []format.Format slice:

desc, _, err := c.Describe(u)
if err != nil {
    panic(err)
}

for _, medi := range desc.Medias {
    for _, forma := range medi.Formats {
        // forma is a format.Format interface value
        fmt.Printf("payload type %d  clock rate %d  rtpmap %s\n",
            forma.PayloadType(), forma.ClockRate(), forma.RTPMap())
    }
}

Detecting a specific format

Because each codec is a concrete struct, you can use a type assertion to check for it:

for _, medi := range desc.Medias {
    for _, forma := range medi.Formats {
        if h264, ok := forma.(*format.H264); ok {
            // h264 is now a *format.H264
            fmt.Println("found H264, packetization-mode:", h264.PacketizationMode)
        }
    }
}

The helper desc.FindFormat does this for you and also returns the matching media:

var forma *format.H264
medi := desc.FindFormat(&forma)
if medi == nil {
    panic("no H264 stream found")
}

Creating encoders and decoders

Once you have a concrete format value, call CreateDecoder() to receive an RTP depacketizer, or CreateEncoder() to receive an RTP packetizer. The returned objects are format-specific, with typed Decode / Encode methods.

// Decoder — use while receiving a stream
rtpDec, err := forma.CreateDecoder()
if err != nil {
    panic(err)
}

c.OnPacketRTP(medi, forma, func(pkt *rtp.Packet) {
    au, err := rtpDec.Decode(pkt)
    // au is [][]byte for H264, []byte for Opus, etc.
})

// Encoder — use while sending a stream
rtpEnc, err := forma.CreateEncoder()
if err != nil {
    panic(err)
}

pkts, err := rtpEnc.Encode(accessUnit)
for _, pkt := range pkts {
    c.WritePacketRTP(medi, pkt)
}

Not every format ships with a built-in encoder or decoder. Formats such as Speex, Vorbis, and G726 require you to supply your own RTP packetizer. See the audio and video reference pages for availability details.

Next steps

Video formats

H264, H265, AV1, VP8, VP9, M-JPEG, MPEG-4 Video, and MPEG-1/2 Video.

Audio formats

Opus, MPEG-4 Audio (AAC), G711, G722, LPCM, AC-3, Speex, and more.