New: Cekura Voice AI BenchmarksView results

WebRTC vs WebSocket for Voice AI, Compared Hop by Hop

Janhvi Nandwani
Written byNOV 20, 202619 MIN READ
Janhvi NandwaniinExpert verified
Founding Member, CekuraIIT Bombay (B.Tech, Mech)

Has stress-tested 5M+ voice agent minutes at Cekura.

I read the RFCs behind both protocols and the transport docs of 7 voice AI vendors to settle WebRTC vs WebSocket for voice agents.

Production agents usually run both, with WebRTC carrying live audio to the caller's device and WebSocket carrying streams between your servers and the speech APIs.

I checked those docs on November 11, 2026, and didn't run a packet-loss test of my own. So every number below comes from the spec, the vendor, or the benchmark that published it.

WebRTC vs WebSocket: What's the Difference?

The main difference is what happens when a packet drops. WebRTC conceals the gap and keeps the audio playing, while WebSocket waits for TCP to resend it.

WebRTC is a set of browser APIs and IETF protocols that sends encrypted audio and video live, usually over UDP.

WebSocket is a two-way message channel between a client and a server that, in its classic form, runs on a single TCP connection.

Choose WebRTC if: audio travels between a browser or mobile app and your agent, over Wi-Fi or cellular networks you don't control.

Choose WebSocket if: you’re streaming audio or text between servers (a telephony media stream, a TTS API), you need a channel for signaling and control messages, or you’re building chat, live dashboards, or other apps that need every message delivered in order.

The two also lean on each other. The W3C WebRTC spec leaves signaling to a channel "provided by unspecified means" and names WebSocket as an example, so plenty of WebRTC apps open a WebSocket first to trade connection details.

Meet WebRTC: Key Features

WebRTC is a W3C Recommendation (the current version is dated March 13, 2025), and the IETF published most of its protocol suite as RFCs in January 2021. Under the API are RTP for media, ICE for finding a network path, and DTLS-SRTP for encryption.

UDP is the default transport. RFC 8835 describes UDP as the protocol WebRTC's protocol elements generally assume, and section 1 below shows what that means when a packet drops.

For firewalls that block UDP, the same RFC requires every WebRTC endpoint to support TURN relays over TCP and over TLS. It also notes that TCP between the TURN server and the peer “may result in more performance problems than using UDP, e.g., due to head of line blocking.”

The audio stack ships with the protocol. RFC 7874 requires WebRTC endpoints to implement Opus and G.711.

The open-source WebRTC voice engine adds NetEQ, which its architecture page describes as a jitter buffer that also conceals lost packets. The same engine includes an acoustic echo canceler.

Opus covers a wide range. RFC 6716 puts it at 6 kbit/s to 510 kbit/s, with frames from 2.5 to 60 ms and in-band forward error correction for lost packets.

WebRTC also has data channels for non-media messages. RFC 8831 runs them on SCTP over DTLS and lets each message be sent "ordered or unordered and with partial or full reliability."

Meet WebSocket: Key Features

WebSocket is older and much smaller. RFC 6455, published in December 2011, defines it as "an opening handshake followed by basic message framing, layered over TCP."

The connection starts as an HTTP Upgrade request on port 80 or 443. That's why WebSocket gets through the proxies and firewalls that already allow web traffic, though some corporate proxies still block the Upgrade.

Later RFCs also let it run inside an HTTP/2 connection or an HTTP/3 one.

After the handshake, either side can send text or binary frames at will. TCP delivers every byte in order, which is exactly what you want for JSON events, transcripts, and LLM tokens.

WebSocket has no idea what audio is. You get no codec, and no jitter buffer, and echo cancellation is on you as well. You pick the format and chunk the audio, then time playback yourself.

Twilio Media Streams shows what that looks like in practice. Its docs fix the format at audio/x-mulaw with a sample rate of 8000 Hz, sent as base64 inside JSON messages.

Base64 adds a third to the payload. By my arithmetic, 64 kbit/s of G.711 audio becomes about 85 kbit/s before protocol overhead is counted.

Opus can encode speech at a fraction of that bitrate, though the total on the wire also depends on packet size and RTP overhead.

There's also a browser-side limit. MDN notes that the WebSocket interface "doesn't support backpressure," so a client that receives audio faster than it can process it will buffer without bound.

WebRTC vs WebSockets: At a Glance

The technical rows come from the RFCs, the W3C spec, and MDN, and the “Best for,” “Key strength,” and “Main weakness” rows sum up the sections below.

FeatureWebRTCWebSocket
Best forLive audio between a user's device and the agentServer-to-server streams, signaling, text and events
TransportUDP by default; TURN over TCP or TLS as fallbackOne TCP connection in classic RFC 6455 (TLS for wss://)
Connection modelPeer-to-peer by design; voice agents usually connect through a media serverClient to server
Under packet lossOver UDP, lost audio is concealed and playback continuesThe stream waits until TCP resends the lost segment
EncryptionMandatory; RFC 8827 bans unencrypted RTPOptional; ws:// is plaintext, wss:// adds TLS
NAT traversalICE with STUN and TURN serversNo ICE, STUN, or TURN; the client opens an outbound connection, usually on port 443
Browser supportWell supported in modern browsers, with "some incompatibilities" per MDNWidely available across browsers since July 2015
Audio processingOpus and G.711, jitter buffer, and echo cancellation includedNone; you build or import it
SignalingNot standardized; often done over WebSocketThe HTTP Upgrade handshake is the setup
PricingFree open standard; you run or rent TURN and media serversFree open standard; runs on standard web servers
Key strengthKeeps audio flowing on lossy networksSimple to deploy and supported by nearly every speech API
Main weaknessMore moving parts to set up and debugTCP head-of-line blocking stalls live audio

WebRTC vs WebSocket: Feature-by-Feature Comparison

1. Transport and Packet Loss

Every other difference grows from this one. WebSocket inherits TCP's promise to deliver every byte in order, and WebRTC media gives that promise up on purpose.

  • WebRTC: RTP runs over UDP, and each packet carries a sequence number and timestamp. If packet 41 never arrives, the receiver plays 40, conceals the missing audio, and plays 42 on schedule.

  • WebSocket: If the TCP segment holding packet 41 is lost, 42 through 60 sit in the kernel buffer until 41 is resent. The audio then arrives in a burst, late.

LiveKit's engineering blog puts numbers on it: "A missing 20ms audio frame is nearly imperceptible to a listener; a 200ms stall while TCP retransmits is not." The 200 ms is an illustration from LiveKit, with no test behind it.

On restrictive networks, WebRTC can lose that edge. When a firewall blocks UDP and WebRTC falls back to TURN over TCP, it picks up TCP’s head-of-line blocking on that leg.

Winner: WebRTC, for live audio that crosses the public internet.

2. Latency and Jitter on a Voice Turn

On a clean connection, the transport is a small slice of the turn. In Cekura's speech-to-speech benchmark, 15 voice models and a cascade reference took 3,936 live calls over WebRTC, and the models' median response times ran from 1.34 to 2.88 seconds.

Set LiveKit's 200 ms stall against that range, and it adds 7% to 15% to one turn, by my arithmetic. So on good Wi-Fi, your STT, LLM, and TTS choices move the number far more than the protocol does.

Bad networks are where the two split. WebRTC's jitter buffer absorbs uneven packet arrival, holding "a few tens of milliseconds of audio" according to Cartesia's guide. On WebSocket, that smoothing is code you write.

The same benchmark groups silence, coughs, and packet loss under one pass rate. Across the 13 models scored there, it averaged 85%, from 100% for both Grok Voice models down to 54% for Deepslate Opal.

Every one of those calls ran on WebRTC, which tells me the protocol alone doesn't make an agent behave well when audio drops out.

The benchmark page doesn't say how packet loss was produced, though. I also found no published run of one agent on both transports, so there's no first-party number for the difference itself.

Distance costs the same on both. LiveKit's post works through a user in Singapore talking to an agent in US-East, and its closing note puts the round trip at roughly 230 to 280 ms before the agent starts processing.

ITU-T G.114 sets 150 ms of one-way delay as the point below which callers barely notice the lag, and it advises against planning any network above 400 ms.

Our voice AI latency guide covers how to track RTT, jitter, and packet loss per region.

Winner: Tie on a clean network. WebRTC once loss and jitter show up.

3. Audio Processing in the Stack

A browser's WebRTC stack handles capture, encoding, echo cancellation, and playback timing before your code sees a frame.

For a voice agent, echo cancellation matters more than the rest. Without it, the agent's own voice leaks from the speaker into the mic and the agent interrupts itself.

Daily's WebRTC primer says echo cancellation comes with the typical WebRTC implementation. LiveKit's post lists acoustic echo cancellation, automatic gain control, and noise suppression as part of the client.

With WebSocket in a browser, you capture audio through the Web Audio API, encode it, ship it, then schedule playback of the chunks that come back. That works for a demo, and I'd budget engineering time for it before trusting it on a speakerphone.

Between servers, none of this applies. A TTS API streaming PCM to your agent process has no microphone or speaker, so WebRTC's audio stack has nothing to do there.

Winner: WebRTC on the client. On a server-to-server hop, the feature doesn't apply.

4. Setup, NAT Traversal, and Servers

One HTTP Upgrade request opens a WebSocket connection, and it works through the same load balancers as the rest of your API.

A production WebRTC deployment usually needs more parts:

  1. A signaling channel to exchange session descriptions (often a WebSocket)

  2. STUN servers so each side can learn its public address

  3. TURN servers to relay media when a direct path fails

  4. A media server or hosted platform between the client and a cloud-hosted agent

RFC 8835 requires a full ICE implementation, and debugging a failed ICE negotiation is slower than reading a WebSocket close code. Cartesia's comparison table lists the server side as usually "a media server or a hosted platform."

Those 4 parts are why frameworks exist at this layer. Pipecat ships a peer-to-peer SmallWebRTC transport and a Daily transport, and LiveKit runs the media server for you.

Winner: WebSocket.

5. Encryption and Browser Support

WebRTC encrypts by default, and WebSocket encrypts when you ask it to. RFC 8827 states that media traffic "MUST NOT be sent over plain (unencrypted) RTP or RTCP" and that implementations must support DTLS-SRTP.

WebSocket leaves it to the URL scheme, where ws:// is plaintext and wss:// runs over TLS. Use wss:// in production and the two end up close in practice, though the plaintext option still exists and WebRTC has no equivalent.

Both run in current major browsers. MDN marks the WebSocket API as widely available since July 2015 and says WebRTC is "in general well supported in modern browsers, but some incompatibilities remain."

Winner: WebRTC on encryption by default, and a tie on browser support.

6. Where Each Protocol Sits in a Voice Agent

A single call can cross WebRTC, WebSocket, and SIP before the caller hears a word. A cascaded voice agent has 3 kinds of hops, and each has a natural protocol.

HopTypical protocolWhy
Browser or mobile app to agentWebRTCLossy last-mile networks, echo cancellation, Opus
Agent to STT, LLM, and TTS providersWebSocket (or HTTP streaming)Data-center links with little loss; nearly every speech API exposes it
Phone caller to agentSIP and RTP, or a carrier's WebSocket media streamThe carrier decides; audio is usually 8 kHz G.711

Browser or app to agent

LiveKit's agent docs say "LiveKit WebRTC is used between the frontend and the agent," while the agent talks to your backend over HTTP and WebSockets. Daily’s developer docs say Daily handles the WebRTC infrastructure for you.

Our LiveKit Agents guide walks through how an agent joins a room.

OpenAI makes the same split for its Realtime API. Its docs recommend WebRTC over WebSockets for browser and mobile clients.

For GPT-Live, OpenAI's getting-started page recommends WebRTC for audio in browser, mobile, and server applications and lists WebSockets as a supported alternative. Its server guide still uses a WebSocket when the server captures or relays the audio.

Agent to providers

Streaming STT and TTS APIs are WebSocket territory. ElevenLabs documents a WebSocket endpoint designed "to generate audio from partial text input."

Cartesia's TTS docs say to use WebSocket when "the full transcript is not known up front," which is the case whenever an LLM streams its answer token by token.

The same Cartesia page adds that reusing one connection across turns avoids extra TCP and TLS round trips, "often tens to low hundreds of ms per turn" by its own estimate.

Phone calls

A phone call usually reaches your agent in one of 2 ways. With SIP trunking, the carrier sends SIP signaling and RTP audio.

LiveKit's telephony docs say inbound calls "automatically create SIP participants that join LiveKit rooms," so a phone caller ends up in the same room as a WebRTC user.

The other way is a media stream. Twilio Media Streams lets you "stream raw audio from live phone calls over WebSockets to your application."

Pipecat's FastAPI WebSocket transport is "primarily designed for telephony integrations with providers like Twilio, Telnyx, and Plivo."

A PSTN leg on G.711 carries 8 kHz narrowband audio, whatever sits behind it. G.711 passes only 300 to 3,400 Hz, which is why an agent that sounds sharp in a 16 kHz browser demo can mishear digits on a phone call.

Winner: Depends on the hop, and the table above maps each one.

What Voice AI Vendors Say About WebRTC and WebSocket

These quotes come from vendor docs and engineering blogs, checked word for word. Each vendor sells something at this layer (LiveKit and Daily sell WebRTC infrastructure, Cartesia and ElevenLabs sell WebSocket speech APIs), so weigh your options with that in mind.

WebRTC

Pros

  • Client audio: OpenAI's Realtime docs say that for a browser or mobile device, "we recommend using WebRTC rather than WebSockets for more consistent performance."

  • Packet loss: In LiveKit’s words, “When a packet is lost, the stream keeps flowing.”

  • Built-in audio processing: LiveKit’s post lists acoustic echo cancellation, automatic gain control, and noise suppression as part of the WebRTC client.

  • Infrastructure: Cartesia’s guide lists the server side as usually “a media server or a hosted platform”.

Cons

  • Setup: Cartesia's guide lists what a connection takes: "Signaling, ICE candidates, often STUN and TURN servers."

  • Browser quirks: MDN says WebRTC is “in general well supported in modern browsers, but some incompatibilities remain,” so test on each browser your users run.

WebSocket

Pros

  • Server-to-server: For its Realtime API models, OpenAI calls WebSockets "a great choice for connecting to the OpenAI Realtime API in server-to-server applications."

  • Telephony: Pipecat's transport guide lists it among its key takeaways: "WebRTC for clients, WebSocket for telephony."

Cons

  • Falling behind: Daily's primer puts it plainly, "A WebSocket client that gets behind sending/receiving data will fall farther and farther behind."

  • Audio over the internet: Daily's LLM voice post gives the opposite advice for media, "Don't use web sockets for audio or video transport (if you can avoid them)."

  • Whole-text TTS: ElevenLabs says WebSocket "may not be the best choice" when the entire input text is available upfront, because the buffering can mean slightly higher latency than a standard HTTP request.

  • Head-of-line blocking: Pipecat's guide flags its WebSocket transport as "TCP-based" and lists head-of-line blocking among its limits.

The vendors agree more than their business models would suggest. Cartesia, which sells a WebSocket API, says production voice agents typically use both.

How to Make Your Choice

The WebRTC vs WebSocket choice gets simpler once you split the call by hop. For audio that crosses the public internet to a person's device, use WebRTC.

Between your own servers and your STT and TTS providers, WebSocket is simpler and costs you little, because data-center links rarely drop packets.

WebRTC Is Better For

  • Web and mobile voice agents where users are on Wi-Fi or cellular

  • Speakerphone and laptop use, where echo cancellation decides whether barge-in works

  • Agents that add video or screen sharing later

  • Speech-to-speech models that accept WebRTC directly, such as OpenAI's

WebSocket Is Better For

  • Streaming STT and TTS connections from your agent server

  • Telephony media streams from Twilio, Telnyx, or Plivo

  • Signaling, tool events, and transcripts

  • Prototypes on a wired or office network, where Pipecat's guide says WebSocket is fine when "latency isn't critical"

My Verdict

My WebRTC vs WebSocket answer for a new voice agent is to use both: WebRTC from the client to a media server, WebSocket from the agent to the STT and TTS providers, and a SIP trunk for phone numbers.

That’s the client-side layout LiveKit’s docs describe, and the split Pipecat’s transport guide recommends, and I found no vendor arguing for the reverse.

Plan for one exception if the model is OpenAI's GPT-Live. Its docs lean toward WebRTC for server-side audio too, so check that before you wire a WebSocket.

If you already run browser audio over WebSocket and your users are on office Wi-Fi, I wouldn't rewrite it this week. I'd first measure how often audio stalls on a throttled mobile connection, because users who call from their phones are the ones who'll hit it.

Next Steps: Test the Transport Layer on a Bad Network

A transport choice that looks fine on your laptop can fail in a parking garage. In Cekura's voice agent benchmark, 10 configurations took 2,460 calls, and the share with no provider or connection issue ran from 100% for Retell, ElevenLabs, and Telnyx down to 72.4% for Gemini Live.

Vapi scored 82.9%, so about one in six of its 246 calls hit a provider or connection issue. Each provider chose its own stack, and the page doesn't name the transport, so treat the scores as relative and confirm them on your own traffic.

LiveKit and Pipecat are in that benchmark and are also Cekura integration partners.

To check your own agent, run the same scenarios over each path a caller can take:

Pre-production

  • Simulated calls per transport: Cekura joins LiveKit and Pipecat agents over WebRTC, dials an agent’s own wss:// WebSocket endpoint, connects to a SIP-based agent, or calls your own Twilio or Plivo numbers.

This way, one test suite covers the browser path and the phone path. The telephony testing guide lists which phone-side failures to test first.

Infrastructure

  • Infrastructure Suite: 18+ ready-made scenarios cover hold tests, interruptions, audio quality, and packet loss.

  • Load testing: Concurrent calls show the volume at which turn latency and error rates start to climb.

Observability

  • Live call scoring: Production calls are scored on latency and interruptions, so a regression on one region or carrier shows up as a metric.

Native integrations cover Retell, Vapi, ElevenLabs, LiveKit, Pipecat, Bland, and more.

Cekura supports SOC 2, HIPAA, and GDPR compliance, covering transcript redaction, role-based access, and audit trails.

Pick the protocol per hop, then prove it under loss before callers do it for you. Book a demo to see your agent tested over WebRTC, WebSocket, SIP, and real phone calls, with packet loss and interruptions in the mix.

Frequently Asked Questions

What is the main difference between WebRTC and WebSocket?

The main difference between WebRTC and WebSocket is the transport. WebRTC sends media over UDP and keeps audio playing when packets are lost, while classic WebSocket runs on TCP and delivers every message in order, even if that means waiting.

Is WebRTC better than WebSockets?

Yes, WebRTC is better than WebSockets for live audio between a user's device and a server, because it conceals packet loss and includes echo cancellation. WebSockets are the better fit for server-to-server streams, signaling, and text, which is why production voice agents usually run both.

Is WebRTC faster than WebSocket?

No, WebRTC isn't meaningfully faster than WebSocket on a clean network, since STT, LLM, and TTS time dominates a voice turn. Under packet loss, WebRTC keeps playing while a WebSocket stream waits for TCP to resend, and that wait shows up as a stall in the audio.

What are the downsides of WebRTC?

The downsides of WebRTC are setup and operations. You need a signaling channel, STUN and TURN servers, and usually a media server, and behavior still varies between browsers.

Are WebSockets deprecated?

No, WebSockets aren't deprecated. The WebSocket API is a WHATWG Living Standard that's still being updated, and MDN lists it as widely available across browsers since July 2015.

Does ChatGPT use SSE or WebSocket?

OpenAI's docs don't say whether the ChatGPT app uses SSE or WebSocket. Its API streams text responses over server-sent events (SSE) by default, per its streaming guide, and also offers a WebSocket mode for the Responses API.

How do you test WebRTC vs WebSockets for a voice agent?

To test WebRTC vs WebSockets, run the same call scenarios over each transport with packet loss, jitter, and interruptions added, then compare turn latency and task completion. Only a bad-network run separates the two, since both protocols look alike on a clean connection.

Is WebRTC based on WebSocket?

No, WebRTC isn’t based on WebSocket. They’re separate protocols, but the W3C WebRTC spec leaves signaling to “unspecified means” and names WebSocket as an example, so many WebRTC apps open a WebSocket first to exchange connection details.

Ready to ship voice
agents fast? 

Book a demo