ONVIF back channels

Two-way audio with RTSP back channels for IP cameras.

A back channel is a sendonly media track in the SDP that flows in the reverse direction — from the client to the server. ONVIF IP cameras use back channels to receive audio from a viewer (for example, to drive a speaker mounted on the camera). The codec is almost always G.711 (PCMU or PCMA) at 8 kHz.

The ONVIF Streaming Specification requires cameras to advertise a back channel in the SDP when two-way audio is supported. gortsplib handles back channels on both the client side (sending audio to a camera) and the server side (receiving audio from a viewer).

Back channel media entries have IsBackChannel: true in description.Media. The SDP attribute a=sendonly is set on these tracks; the camera receives rather than sends on them.

Client: sending audio to a camera

To send audio through a back channel, set RequestBackChannels: true on the client before connecting. After Describe, iterate the session medias to find a back channel that uses the codec you want to send (G.711 in this case), then call Setup on it and use c.WritePacketRTP to push encoded audio frames.

1. Enable back channel negotiation

   Set RequestBackChannels: true on the gortsplib.Client. Without this flag the client will not include the ONVIF Require: www.onvif.org/ver20/backchannel header and the camera will not expose back channel medias in the SDP.

2. Find the G.711 back channel

   After Describe, walk desc.Medias looking for entries where media.IsBackChannel == true and the format list contains *format.G711.

3. Setup and play

   Call c.Setup(desc.BaseURL, medi, 0, 0) for the back channel media only, then call c.Play(nil).

4. Encode and write

   Use forma.CreateEncoder() to get an RTP encoder for G.711. On each tick, generate PCM samples, encode them to G.711 (µ-law or A-law), packetise with the encoder, set the timestamp field, then call c.WritePacketRTP(medi, pkt) for each packet.

main.go

// Package main contains an example.
package main

import (
	"crypto/rand"
	"log"
	"time"

	"github.com/bluenviron/gortsplib/v5"
	"github.com/bluenviron/gortsplib/v5/pkg/base"
	"github.com/bluenviron/gortsplib/v5/pkg/description"
	"github.com/bluenviron/gortsplib/v5/pkg/format"
	"github.com/bluenviron/mediacommon/v2/pkg/codecs/g711"
	"github.com/pion/rtp"
)

// This example shows how to:
// 1. generate a dummy G711 audio stream.
// 2. connect to a RTSP server, find a back channel that supports G711.
// 3. route the G711 stream to the channel.

func multiplyAndDivide(v, m, d int64) int64 {
	secs := v / d
	dec := v % d
	return (secs*m + dec*m/d)
}

func randUint32() (uint32, error) {
	var b [4]byte
	_, err := rand.Read(b[:])
	if err != nil {
		return 0, err
	}
	return uint32(b[0])<<24 | uint32(b[1])<<16 | uint32(b[2])<<8 | uint32(b[3]), nil
}

func findG711BackChannel(desc *description.Session) (*description.Media, *format.G711) {
	for _, media := range desc.Medias {
		if media.IsBackChannel {
			for _, forma := range media.Formats {
				if g711, ok := forma.(*format.G711); ok {
					return media, g711
				}
			}
		}
	}
	return nil, nil
}

func main() {
	// parse URL
	u, err := base.ParseURL("rtsp://myuser:mypass@localhost:8554/mystream")
	if err != nil {
		panic(err)
	}

	c := gortsplib.Client{
		Scheme:              u.Scheme,
		Host:                u.Host,
		RequestBackChannels: true,
	}

	// connect to the server
	err = c.Start()
	if err != nil {
		panic(err)
	}
	defer c.Close()

	// find published medias
	desc, _, err := c.Describe(u)
	if err != nil {
		panic(err)
	}

	// find the back channel
	medi, forma := findG711BackChannel(desc)
	if medi == nil {
		panic("back channel not found")
	}

	// setup a single media
	_, err = c.Setup(desc.BaseURL, medi, 0, 0)
	if err != nil {
		panic(err)
	}

	// start playing
	_, err = c.Play(nil)
	if err != nil {
		panic(err)
	}

	// setup G711 -> RTP encoder
	rtpEnc, err := forma.CreateEncoder()
	if err != nil {
		panic(err)
	}

	start := time.Now()
	prevPTS := int64(0)

	randomStart, err := randUint32()
	if err != nil {
		panic(err)
	}

	// setup a ticker to sleep between writings
	ticker := time.NewTicker(100 * time.Millisecond)
	defer ticker.Stop()

	for range ticker.C {
		// get current timestamp
		pts := multiplyAndDivide(int64(time.Since(start)), int64(forma.ClockRate()), int64(time.Second))

		// generate dummy LPCM audio samples
		samples := createDummyAudio(pts, prevPTS)

		// encode samples with G711
		if forma.MULaw {
			samples, err = g711.Mulaw(samples).Marshal()
			if err != nil {
				panic(err)
			}
		} else {
			samples, err = g711.Alaw(samples).Marshal()
			if err != nil {
				panic(err)
			}
		}

		// generate RTP packets from G711 samples
		var pkts []*rtp.Packet
		pkts, err = rtpEnc.Encode(samples)
		if err != nil {
			panic(err)
		}

		log.Printf("writing RTP packets with PTS=%d, sample size=%d, pkt count=%d", prevPTS, len(samples), len(pkts))

		// write RTP packets to the server
		for _, pkt := range pkts {
			pkt.Timestamp += uint32(int64(randomStart) + prevPTS)

			err = c.WritePacketRTP(medi, pkt)
			if err != nil {
				panic(err)
			}
		}

		prevPTS = pts
	}
}

Server: receiving audio from a client

To accept back-channel audio from a connecting viewer, declare a description.Media with IsBackChannel: true alongside your normal (direct) media tracks. Register a packet handler inside OnPlay using ctx.Session.OnPacketRTPAny — this fires whenever the client pushes audio into the back channel.

1. Declare a back channel media

   Add a description.Media entry with IsBackChannel: true to the description.Session. The SDP will contain a=sendonly on that track, signalling to connecting clients that they should send audio on it.

2. Create the ServerStream

   Pass the description.Session (containing both the direct and back channel medias) to gortsplib.ServerStream and call Initialize().

3. Handle inbound packets in OnPlay

   In the OnPlay handler, call ctx.Session.OnPacketRTPAny to register a per-session callback. Use ctx.Session.PacketPTS to decode the presentation timestamp from each incoming packet.

main.go

// Package main contains an example.
package main

import (
	"log"
	"sync"

	"github.com/bluenviron/gortsplib/v5"
	"github.com/bluenviron/gortsplib/v5/pkg/base"
	"github.com/bluenviron/gortsplib/v5/pkg/description"
	"github.com/bluenviron/gortsplib/v5/pkg/format"
	"github.com/pion/rtp"
)

// This example shows how to:
// 1. create a RTSP server which accepts plain connections.
// 2. create a stream with an audio direct channel and an audio back channel.
// 3. write the audio direct channel to readers, read the back channel from readers.

type serverHandler struct {
	server *gortsplib.Server
	stream *gortsplib.ServerStream
	mutex  sync.RWMutex
}

// called when a connection is opened.
func (sh *serverHandler) OnConnOpen(_ *gortsplib.ServerHandlerOnConnOpenCtx) {
	log.Printf("conn opened")
}

// called when a connection is closed.
func (sh *serverHandler) OnConnClose(ctx *gortsplib.ServerHandlerOnConnCloseCtx) {
	log.Printf("conn closed (%v)", ctx.Error)
}

// called when a session is opened.
func (sh *serverHandler) OnSessionOpen(_ *gortsplib.ServerHandlerOnSessionOpenCtx) {
	log.Printf("session opened")
}

// called when a session is closed.
func (sh *serverHandler) OnSessionClose(_ *gortsplib.ServerHandlerOnSessionCloseCtx) {
	log.Printf("session closed")
}

// called when receiving a DESCRIBE request.
func (sh *serverHandler) OnDescribe(
	_ *gortsplib.ServerHandlerOnDescribeCtx,
) (*base.Response, *gortsplib.ServerStream, error) {
	log.Printf("DESCRIBE request")

	sh.mutex.RLock()
	defer sh.mutex.RUnlock()

	return &base.Response{
		StatusCode: base.StatusOK,
	}, sh.stream, nil
}

// called when receiving a SETUP request.
func (sh *serverHandler) OnSetup(
	_ *gortsplib.ServerHandlerOnSetupCtx,
) (*base.Response, *gortsplib.ServerStream, error) {
	log.Printf("SETUP request")

	sh.mutex.RLock()
	defer sh.mutex.RUnlock()

	return &base.Response{
		StatusCode: base.StatusOK,
	}, sh.stream, nil
}

// called when receiving a PLAY request.
func (sh *serverHandler) OnPlay(ctx *gortsplib.ServerHandlerOnPlayCtx) (*base.Response, error) {
	log.Printf("PLAY request")

	// called when receiving a RTP packet
	ctx.Session.OnPacketRTPAny(func(m *description.Media, _ format.Format, pkt *rtp.Packet) {
		// decode timestamp
		pts, ok := ctx.Session.PacketPTS(m, pkt)
		if !ok {
			return
		}

		log.Printf("inbound RTP packet with PTS=%v size=%v", pts, len(pkt.Payload))
	})

	return &base.Response{
		StatusCode: base.StatusOK,
	}, nil
}

func main() {
	h := &serverHandler{}

	// prevent clients from connecting to the server until the stream is properly set up
	h.mutex.Lock()

	// create the server
	h.server = &gortsplib.Server{
		Handler:           h,
		RTSPAddress:       ":8554",
		UDPRTPAddress:     ":8000",
		UDPRTCPAddress:    ":8001",
		MulticastIPRange:  "224.1.0.0/16",
		MulticastRTPPort:  8002,
		MulticastRTCPPort: 8003,
	}

	// start the server
	err := h.server.Start()
	if err != nil {
		panic(err)
	}
	defer h.server.Close()

	// create a RTSP description
	desc := &description.Session{
		Medias: []*description.Media{
			// direct channel
			{
				Type: description.MediaTypeAudio,
				Formats: []format.Format{&format.G711{
					PayloadTyp:   8,
					MULaw:        false,
					SampleRate:   8000,
					ChannelCount: 1,
				}},
			},
			// back channel
			{
				Type:          description.MediaTypeAudio,
				IsBackChannel: true,
				Formats: []format.Format{&format.G711{
					PayloadTyp:   8,
					MULaw:        false,
					SampleRate:   8000,
					ChannelCount: 1,
				}},
			},
		},
	}

	// create a server stream
	h.stream = &gortsplib.ServerStream{
		Server: h.server,
		Desc:   desc,
	}
	err = h.stream.Initialize()
	if err != nil {
		panic(err)
	}
	defer h.stream.Close()

	// create audio streamer
	r := &audioStreamer{stream: h.stream}
	r.initialize()
	defer r.close()

	// allow clients to connect
	h.mutex.Unlock()

	// wait until a fatal error
	log.Printf("server is ready on %s", h.server.RTSPAddress)
	panic(h.server.Wait())
}

ONVIF compliance notes

The ONVIF Streaming Specification (section 5.1.1) requires:

• The Require: www.onvif.org/ver20/backchannel header in the client’s DESCRIBE request — gortsplib sends this automatically when RequestBackChannels: true.
• A sendonly attribute on the back channel m= line in the SDP — gortsplib sets this from IsBackChannel: true in the media description.
• The back channel to be set up and played within the same RTSP session as the forward channel.

To test your implementation without a physical camera, use MediaMTX. It supports ONVIF back channels and can act as both the upstream camera and a test client in a loopback configuration.

Some cameras time out the back channel if no RTP packets are received for several seconds. Send silence (zero-amplitude G.711 frames) periodically as a keepalive if your application does not have continuous audio to transmit.