WebRTC E2EE with SFrame: what your SFU can still read

You shipped end-to-end encryption for your WebRTC calls. The frames are encrypted in a worker before they reach the packetizer, the keys never leave the clients, the little padlock is in the UI and the sales page says the server cannot see your meetings.

Your server operator can still produce, for any call, a timeline of who spoke, when they started, when they stopped, how loudly, and who cut across whom. Not by breaking anything. By reading the fields the conference needs in order to work.

Where Encoded Transform encrypts

The Encoded Transform spec, the API many people still call Insertable Streams, is precise about the seam:

This API allows manipulation of encoded frames in the media pipeline between the processing steps of an RTCRtpSender’s underlying encoder and packetizer, and/or between an RTCRtpReceiver’s underlying depacketizer and decoder.

Between the encoder and the packetizer. Your code gets the encoded frame, encrypts the payload bytes, and hands them back. The packetizer then does its job: splits the frame across RTP packets, writes the header, writes the header extensions, and passes it all to SRTP. None of that is reachable from JavaScript, and the spec does not offer a way to reach it.

So the boundary is not “the server sees nothing”. The boundary is “the server sees everything the RTP layer carries, and nothing inside the payload”. Worth knowing before you write the marketing copy. The API has been Baseline since October 2025, so the hard part is no longer availability.

SFrame leaves metadata readable on purpose

RFC 9605 is honest about it in the first sentence of the abstract:

This document describes the Secure Frame (SFrame) end-to-end encryption and authentication mechanism for media frames in a multiparty conference call, in which central media servers (Selective Forwarding Units or SFUs) can access the media metadata needed to make forwarding decisions without having access to the actual media.

An SFU that cannot read metadata cannot forward. It has to know which layer a packet belongs to, whether a frame is a keyframe, what depends on what, and who is talking loudly enough to be worth sending to forty people. Encrypt all of that and you have not built a secure conference, you have built a broken one.

The costs are modest and predictable. Around 22 bytes per encrypted unit for the config byte, the key id, the counter and the auth tag. The SFrame header itself gets integrity protection but not confidentiality, so the key id and counter are readable by design. And the part that will actually consume your quarter is one sentence in the spec: “the specifics of key management are beyond the scope of this document.” SFrame is the easy half.

The audio level header extension is in the clear

Here is the leak that surprises people, and it predates E2EE by a decade.

RFC 6464 defines a header extension carrying the sender’s own audio level, negotiated as urn:ietf:params:rtp-hdrext:ssrc-audio-level. Seven bits of level, 0 to 127, representing 0 to -127 dBov, plus a voice activity flag. Browsers offer it by default, and every SFU uses it, because the alternative is decoding every stream in the conference to find the active speaker.

On a 20ms audio packetization interval that is fifty readings per second, per participant, of how loud each person is. RFC 6464’s own security considerations say what that means:

audio levels are therefore visible on a packet-by-packet basis to an attacker passively observing the audio stream

and go on to note that the pattern is enough to infer phonemes. That warning is about an attacker who is merely watching the wire. Your SFU is not watching the wire. It terminates SRTP, so it holds the keys and reads the extension as a field.

Encrypting the Opus payload does not touch any of this. You get a speaker diarisation log for free, with silence, overlap and interruption all clearly visible, and the words themselves safely encrypted inside.

Other metadata the SFU still sees

The list of what the pipeline needs in the clear is roughly the list of what your transform receives as metadata, because the browser hands JavaScript the same fields the packetizer is about to write.

The SSRC identifies the stream and the MID and RID identify which simulcast layer it is, which is a direct read on resolution. Transport-wide sequence numbers and send times are what the bandwidth estimator runs on. A dependency descriptor carries the frame id and dependency structure for SVC layer selection. Frame boundaries and keyframe flags have to survive, because the SFU decides what to forward on exactly that basis.

Then there is the shape of the traffic, which nobody negotiates and nobody can turn off. Video frame sizes track motion, so a still talking head and someone gesturing at a whiteboard are not hard to tell apart. Keyframe timing marks joins and layer switches. Encrypted payloads do not pad themselves, so the byte counts are as informative as they ever were.

Cryptex encrypts header extensions, but not from the SFU

RFC 9335 is the fix for header extensions in general. It encrypts extension ids, lengths and values, plus CSRCs, negotiated with a bare a=cryptex attribute, and it replaces RFC 6904’s per-extension signalling with something people will actually deploy. It is in libwebrtc, so Chromium negotiates it when the other side offers it.

Turn it on. It closes the passive-observer case that RFC 6464 warns about, and it costs you nothing.

It does not help against your server. SRTP in a conference is hop-by-hop, and the SFU is the hop: it decrypts what you sent and re-encrypts for each receiver, which is the whole reason DTLS-SRTP alone was never end-to-end. Cryptex changes who can read your header extensions on the wire. The SFU was never on the wire, it was the other end of it.

Turning off the audio level extension

Stop negotiating the extensions your server does not need. This is the legitimate use of setHeaderExtensionsToNegotiate() on the transceiver, and it is also the one case where the thing you want is an explicit API rather than an edit to the SDP.

const exts = transceiver.getHeaderExtensionsToNegotiate();
for (const ext of exts) {
  if (ext.uri === 'urn:ietf:params:rtp-hdrext:ssrc-audio-level') ext.direction = 'stopped';
}
transceiver.setHeaderExtensionsToNegotiate(exts);

Be clear-eyed about the bill. Without the audio level, your SFU cannot pick an active speaker, which means it cannot decide whose video to forward in a large call, which is a feature you probably sell. You can move speaker detection to the clients, but the clients only receive what the server already chose to send, so on a call of any size you are choosing between the metadata leak and the product. Most teams will keep the leak, and that is a defensible decision as long as somebody made it on purpose.

Padding to a constant rate defeats the size analysis and costs you bandwidth on every stream to hide something most threat models do not include. Recording, server-side transcription, captions and mixing all stop working the moment the payload is opaque, so if any of those are in the product, E2EE is a mode, not a default.

What you can honestly claim

The claim you can defend is that the server cannot hear the words. It is a real and worthwhile property and SFrame delivers it.

The claim that the server learns nothing is not true of any SFU-based conference, encrypted or otherwise, and the fields that give it away are documented, negotiated in your own SDP, and warned about in a 2011 RFC. Write that in the security page instead of the padlock. The people who read security pages already know, and they will trust the rest of it more.