Echo cancellation is the signal processing that removes a device's own speaker output from its microphone feed before that audio re-enters a call. In voice AI, it keeps an agent from transcribing its own synthesized speech as caller speech, which is the difference between a working agent and one that interrupts itself.
Every voice agent plays audio and listens at the same time. That is a feedback loop waiting to happen, and the component that breaks the loop is an acoustic echo canceller. When it works, nobody notices. When it degrades, the agent starts answering itself, and the transcript fills with the agent's own words attributed to the caller.
What does echo cancellation do to a microphone signal?
A microphone does not know which sounds matter. It captures whatever reaches it, including audio the same device just played through its own speaker. On a call, that captured speaker audio travels back to the person who originally spoke, and they hear themselves a fraction of a second late.
The W3C Media Capture and Streams specification puts it this way: "When one or more audio streams are being played in the processes of various microphones, it is often desirable to attempt to remove sound being played from the input signals recorded by the microphones."
An echo canceller does this by subtraction, not by muting. It has access to two signals:
- The reference signal. The audio the device is about to play, known exactly, because the device generated it.
- The microphone signal. Whatever the microphone actually picked up, which contains the caller's speech plus a delayed, distorted copy of the reference.
The canceller runs an adaptive filter that continuously estimates the echo path, meaning everything the room and the hardware did to the reference signal on its way from speaker to microphone. It then subtracts that estimate from the microphone signal. What survives the subtraction is treated as real speech.
The classic implementation is a normalized least mean squares (NLMS) adaptive filter, and the time it takes to settle on a good estimate is its convergence time. Production cancellers pair it with a double-talk detector that freezes adaptation while the near-end speaker is talking, so the filter does not learn from a signal it cannot separate.
Two things make this hard. The echo path changes constantly as people move and hold devices differently, so the filter has to keep re-estimating. And during double talk, when both sides speak at once, the filter can mistake real speech for echo and adapt in the wrong direction, cutting into the words it was supposed to preserve.
A linear adaptive filter also only removes the part of the echo it can model. The ICASSP challenge paper notes that the performance of signal processing based cancellers degrades when their assumptions are violated, giving fast time-varying acoustics, unknown signal processing blocks, and non-linearities in the processing chain as examples. Loudspeaker distortion is a common example of a non-linearity a linear filter cannot model, and what it leaves behind is called residual echo. That is why entries to the challenge series have ranged from pure deep models through a linear canceller paired with learned echo suppression to conventional signal processing.
What is acoustic echo cancellation, and how is it different from line echo?
Acoustic echo cancellation, usually shortened to AEC, handles echo that travels through the air. Speaker plays, room reflects, microphone captures. It is the dominant problem on speakerphones, laptops, and any headset-free setup.
Line echo is different in origin. It comes from impedance mismatch in analog telephony, where a hybrid circuit converting between two-wire and four-wire paths reflects part of the signal back down the line. Voice agents connected to the public telephone network can meet both kinds on the same call.
The distinction matters because the fix lives in a different place. Acoustic echo is best cancelled at the device that owns both the speaker and the microphone, because that device holds the reference signal with the shortest and most stable echo path. A server-side agent also holds a reference, its own synthesized speech, but sees the echo through network delay and jitter, which is what makes server-side cancellation the harder problem.
Cancellers are specified by tail length, the span of echo path delay the adaptive filter can model. Line echo from a nearby hybrid returns almost at once. Acoustic echo keeps arriving for as long as the room reverberates, and the ICASSP 2023 challenge's synthetic rooms use reverberation times from 200 ms to 1,200 ms. A server-side agent hears its own speech return only after a network round trip on top of that, so a canceller sized for a handset can run out of tail behind a SIP trunk. Line echo is normally handled inside the carrier network by a canceller conforming to ITU-T Recommendation G.168. G.131 also warns that at interconnections between public and private networks the public side may not provide sufficient echo control, and puts the burden on the private network provider, which today includes anyone running an agent behind a SIP trunk.
Research attention has stayed on the acoustic side. The Microsoft team behind the ICASSP 2023 Acoustic Echo Cancellation Challenge, the fourth edition of that challenge, put the reason in the first paragraph of their paper:
The degradation of call quality due to acoustic echoes is one of the major sources of poor speech quality ratings in voice and video calls.
Ross Cutler et al., Microsoft Corporation, ICASSP 2023 Acoustic Echo Cancellation Challenge
That challenge is a useful reference point for anyone specifying a canceller. It caps combined algorithmic and buffering latency at 20 ms, and its open training data comes from recordings of more than 10,000 real audio devices and human speakers in real environments, alongside a synthetic set. Both numbers tell you where the research bar stood as of the 2023 challenge.
At what delay does echo cancellation become necessary?
Echo is not always a problem. A short enough round trip and the reflection blends into your own voice, the way it does in an ordinary room. Past a certain delay, your brain separates the two and the reflection becomes a distinct, distracting echo.
ITU-T Recommendation G.131, "Talker echo and its control," fixes where that line sits. Appendix I of the recommendation states that no special echo control devices are needed if the one-way transmission time is under 25 ms, which corresponds to a talker echo loudness rating of 33 dB at that 25 ms limit.
Two caveats belong with that number every time it is used. G.131 notes that the blanket 25 ms deployment rule came from former versions of the recommendation, and the current text says the threshold remains valid for networks using 600 ohm hybrids rather than universally. It also allows that echo control may be deployed below or above that transmission time depending on the loudness ratings expected in a given network. The threshold moves with echo loss: Appendix I works two examples in which the acceptable limit falls to 18 ms at a talker echo loudness rating of 30 dB, and to 9 ms at 24 dB.
For voice agents the practical reading is simple. Almost no real deployment stays under 25 ms one-way. Add network transport, jitter buffering, speech recognition, model inference, and speech synthesis, and the round trip is an order of magnitude larger. Echo control is not optional at that scale, and the same delay budget that governs perceived responsiveness also governs whether leaked audio is audible as echo.
Why do AI voice agents fail differently when echo cancellation breaks?
For a human call, a failed canceller is annoying. You hear yourself, you talk around it, the call continues.
For an AI agent, the same failure ends the conversation, because the agent has no way to recognize its own voice.
The leaked audio reaches voice activity detection as ordinary sound energy. VAD reports speech. The turn detector concludes the caller has started talking. Speech recognition transcribes the agent's own sentence and hands it to the model as a user turn.
The result is a self-sustaining loop. The agent speaks, hears itself, treats that as an interruption, stops mid-sentence, then responds to its own words. A regression in one voice SDK release was reported with exactly this symptom: the agent's speaker output was picked up by the microphone and interpreted as user speech, producing a loop where the agent responded to itself until the call was unusable.
This is why echo cancellation and barge-in handling are the same engineering problem viewed from two sides. Barge-in requires the agent to keep listening while it speaks, so muting the microphone during playback is not available as a fix. The only way to listen during playback without hearing yourself is to subtract the reference signal correctly.
The failure directions are worth naming separately, because they need different tests:
| Failure | What the caller experiences | Root cause |
|---|---|---|
| False barge-in | Agent stops mid-sentence with nobody speaking | Residual echo passes VAD as speech |
| Missed barge-in | Agent talks over a real interruption | Suppression set so aggressively it removes caller speech too |
| Self-response loop | Agent answers its own last sentence | Echo reaches speech recognition and is transcribed as a user turn |
| Clipped caller speech | Words lost when both parties speak | Adaptive filter misconverges during double talk |
Practitioners building agents describe the same trap. One of them, replying to a writeup on building a voice agent from scratch, put the rule as: if you do not subtract your own speech output, or at least gate voice activity detection while it is playing, you will get false user starts. That maps directly onto the first and third rows above, and it is the reason turn detection cannot be tuned in isolation from audio processing.
Echo cancellation software: where it sits in the stack
Teams searching for echo cancellation software rarely end up buying a separate product. Standalone cancellers exist, mostly as licensed DSP libraries and SDKs, but in a voice agent the acoustic echo cancellation software you actually depend on is usually a layer inside something you already run, and which layer depends on how the call arrives.
Browser and app clients. The WebRTC stack implements AEC natively and exposes it as the echoCancellation constraint on an audio track, which an application can set to true, false, or a mode that cancels only remote audio. This is the most common path for web-based agents.
Server-side and telephony paths. Calls arriving over SIP or the public telephone network have no browser doing the work. LiveKit's noise and echo cancellation documentation is explicit that WebRTC cancellation runs in the client only, which means a server-side agent answering a phone call needs cancellation supplied elsewhere, either by the carrier or by a server-side model.
Device hardware. Phones and conference room hardware ship their own AEC in the device's audio DSP, upstream of anything your application sees.
The stacking rule matters more than the choice. The same documentation warns against running enhanced noise cancellation models in both the frontend and the agent, because those models are trained on raw audio and behave unpredictably when the input has already been processed. The same page adds that standard WebRTC echo cancellation can stay enabled alongside an agent-side model: the conflict it warns about is between two noise models, not between echo cancellation and noise suppression. The practical rule is to know which stage has already touched the signal before you add another.
How do you test echo cancellation before your callers do?
Echo problems are environmental, which is what makes them slip through review. A developer testing on a headset will never see the speakerphone failure. The bug ships, and it only appears for callers in cars, on speakerphone, or on one carrier route.
Cekura tests this the way the failure actually occurs, by placing real calls through the full stack rather than by inspecting audio in isolation. Because the agent under test is driven by a simulated caller over a real audio path, residual echo either produces a measurable interruption failure or it does not.
The laboratory metric for a canceller is echo return loss enhancement, or ERLE: the ratio in decibels between the echo energy in the microphone signal and the residual echo left after cancellation. It is a weak proxy for what a caller hears. The ICASSP 2023 challenge paper reports a Pearson correlation of just 0.31 between ERLE and subjective P.808 listener ratings on single talk echo scenarios, and notes ERLE is only appropriate in a quiet room with no background noise and only during single talk. That is why the challenge scored entrants on mean opinion score and word accuracy instead, and why a voice agent is better judged on what happened in the call than on a canceller's bench figure.
Three checks are worth automating:
- Silence probes. Run scenarios where the simulated caller says nothing while the agent speaks. Any turn the agent takes against silence is residual echo crossing the VAD threshold, and it is unambiguous evidence.
- Double talk scenarios. Have the caller interrupt at a known offset, repeatedly. Cekura scores whether the interruption was honored, which separates a canceller that is too aggressive from one that is too permissive.
- Transcript self-match. A custom evaluator in Cekura can compare caller-attributed transcript turns against the agent's own prior output. A high similarity score is the self-response loop, caught mechanically before a caller ever meets it.
What does Cekura's benchmark say about interruption handling?
Per Cekura's benchmarks, a frozen matched study of 7 voice agent configurations across 82 scenarios with 3 retained repeats each, interruption handling barely separates platforms: six of the seven configurations score between 4.96 and 5.00 out of 5, and one scores 4.73, measured as the mean interruption score across calls where the interruption evaluator applied, not across all calls in the run. Every provider in that study received the same system prompt, tool definitions, and test data.
The narrowness of that spread is itself the finding. Interruption handling is close to solved under controlled conditions, so a team seeing frequent false interruptions in production should test the audio path before re-tuning turn detection. Cekura's audio quality monitoring exists for that distinction, separating network and codec degradation from pipeline behavior on live calls.
Cekura also runs these scenarios on a schedule against the live agent, and scores real production calls with the same metrics, rather than checking only pre-release builds, because echo regressions arrive through dependency updates and carrier routing changes that no code review catches. When a production call fails one of these checks, Cekura reproduces it in simulation as a repeatable scenario, so the regression becomes a test rather than a support ticket.
The three checks above fit naturally into a Cekura regression suite gated on every deploy, which is the cheapest place to catch an echo regression. If you want to see how your own agent behaves on the silence probe, Cekura can run it against your agent with the same interruption evaluator used in the benchmark.
Frequently asked questions about echo cancellation
Does echo cancellation add latency to a call?
Yes, a small amount. The ICASSP 2023 Acoustic Echo Cancellation Challenge capped combined algorithmic and buffering latency at 20 ms for entrants, which indicates the range modern cancellers target. That is minor next to speech recognition and model inference, but it is not zero and belongs in a delay budget.
Why does my voice agent hear itself even with AEC enabled?
Usually because the canceller is running in the wrong place. WebRTC cancellation runs client side, so a server-side agent on a SIP or telephony path gets no protection from it. Check where in your stack the reference signal is actually available.
What is the difference between echo cancellation and noise suppression?
AEC removes a known signal, your own playback, using that signal as a reference. Noise suppression removes unknown sound, such as traffic or keyboards, using a statistical model. They solve different problems and are usually separate stages.
Can I just mute the microphone while the agent is speaking?
It stops echo, but it also removes barge-in, since a muted microphone cannot hear an interruption. Half duplex behavior is a worse experience for callers than a well-tuned canceller, because every interruption has to wait for the agent to finish, so it is a fallback rather than a fix.
How do I know whether echo is causing my false interruptions?
Run a scenario where the caller stays silent through an entire agent turn. If the agent stops or responds anyway, residual echo is crossing your voice activity detection threshold. That test isolates the audio path from turn detection logic.
What is the difference between echo suppression and echo cancellation?
Echo suppression is the older technique. It detects which side is talking and attenuates the other direction, which makes the call half duplex and clips double talk. Echo cancellation subtracts an estimate of the echo instead, and leaves both directions open. ITU-T G.131 records that echo suppressors built to ITU-T G.164 may still be in use but are not recommended for any new deployment, and directs new deployments to cancellers conforming to G.168.
