How to Handle LLM Stalls and Timeouts in Voice AI Agents

Handling LLM stalls in a voice AI agent means timing the first token rather than the whole reply, then racing a proven backup instead of retrying the same model. Cekura bounds both halves, detection and recovery, so a stalled turn costs a known few seconds instead of an open-ended wait.

By Dileep Chagam, Founding Engineer at Cekura

Last updated: August 2026

TL;DR

• A full-response timeout cannot tell a slow model from a dead one, so Cekura measures liveness by timing the first token instead.
• Cekura sets the first-token threshold at two seconds, because healthy turns produce a first token in about one second.
• Retrying a model that already returned nothing rarely recovers the turn, so Cekura moves on instead of re-asking.
• Cekura races the same request across several providers at once and takes the first to start talking, which bounds recovery by the fastest backup rather than a chain of sequential retries.
• Racing needs a referee that admits exactly one reply, because two providers reaching the caller is worse than the stall that caused them.

What is a silent LLM stall in a voice agent?

A silent LLM stall is a turn where the request is accepted, the connection stays open, every health check passes, and not one token ever arrives. Cekura treats it as the dangerous failure mode in voice, because there is nothing to catch. An outright error is easy. A rate limit or a 500 throws, and the pipeline switches providers. A stall throws nothing, so the only symptom is the clock running down.

On a voice call the thing being defended is timing, not correctness. A caller has a stopwatch running in their head. Answer in about three seconds and the exchange feels natural. Take six and the caller says "hello?", talks over the agent, or hangs up, before they have judged the answer at all.

That clock is not the agent's to control. A voice turn is a chain of steps and several links are hosted services, the language model among them. Deepgram documents streaming transcripts returning in under 300 milliseconds, and Artificial Analysis publishes a Time To First Answer Token figure per model, where the fastest entries sit near 0.3 seconds. A stalled model turn dominates a budget the other layers share. Averages do not hang up on you. Tails do.

What does a stall look like in production?

Cekura measures at the level of the spoken turn, meaning one exchange in which the agent owes the caller a reply, across a fourteen-day window on the production deployment. The table below describes the shape of the problem after the fix shipped, not a before-and-after.

Every figure in this table comes from Cekura's pipecat-cekura-bot production telemetry and carries one caveat that travels with it everywhere it is quoted: the instrumentation postdates the fix it measures, so there is no queryable pre-rollout baseline. The comparison against an open-ended wait describes the previous fixed-timeout implementation as it was written, not a measured before-and-after.

Read the stall row as the number that drives capacity planning, since it sets how often duplicate requests get billed, and read the recovery rows as the bound on what a caller actually experiences once a race fires.

Why does a full-response timeout fail to catch a stalled model?

A full-response timeout is a timer on the whole reply, and Cekura found it has two blind spots that make it the wrong signal for voice.

It fires too late. The alarm only trips once the entire budget has burned, by which point the caller has already gone.

It cannot tell slow from dead. A model that started fast and finished slow is fine. A model that never spoke at all is a disaster. To a completion timer the two are identical.

The signal that matters is liveness, not completion. The useful question is not how long the whole answer is taking, it is how long the agent has said nothing at all. Cekura measures liveness by timing the first token, the earliest available proof that the model is alive. Pipecat exposes this class of measurement directly, documenting a UserBotLatencyObserver for user-to-bot response latency and a TurnTrackingObserver for conversation turns and events. LiveKit documents observability covering transcripts, traces, logs and audio recordings rather than turn-level timing, so a first-token clock on that stack is something you instrument yourself.

How does a first-token watchdog bound the wait?

Cekura's first-token watchdog starts a clock when the agent needs to speak and stops it the instant the first token lands. Nothing within two seconds, and Cekura treats the turn as stalled and acts.

Two seconds is a deliberate threshold, not a round number. Cekura's telemetry records healthy first tokens arriving in about one second, with 99.8% landing under two seconds, on instrumentation that postdates the fix it measures and carries no pre-rollout baseline. The alarm therefore rarely fires on a good call, and the caller barely registers the pause before recovery starts.

Two guards keep the threshold honest. A slow finish never trips it, because the first token already stopped the clock, so a reply that starts fast and ends slow runs to completion untouched. Chosen silence is exempt as well. A pre-recorded message or a hang-up already in progress is not a stall, so the agent never talks over its own goodbye.

Why does retrying the same model rarely recover a stalled turn?

Cekura's production data shows that re-running the same model on an empty reply seldom produces one. Across many calls, same-model recovery stayed low, and each retry stacked more silence onto a turn that had already failed.

So Cekura removed the retry loop. An empty reply now means move on, not re-ask. A model that has already gone silent is a path that has spent its budget, and spending more of the caller's time on it buys nothing back.

The reasoning generalizes past voice. Every retry and every fallback spends something real. A mechanism that does not earn that spend back should be cut rather than tuned. The same discipline governs how Cekura defines and tracks voice AI evaluation metrics: measure what the caller actually experiences, and delete the safety net that only adds delay.

Why race providers instead of failing over to a backup?

Cekura races providers rather than failing over, because fast detection is wasted if recovery is itself unbounded. Serial failover notices the primary stalled and then asks a backup, which means the caller waits on a second request that can also stall, and pays for both in sequence.

Racing removes the sequence. The instant the watchdog fires, Cekura sends the same request to several providers at once, the first to start talking wins the turn, and the rest are discarded. Recovery is bounded by the fastest available backup instead of a chain of sequential retries.

Cekura's telemetry across roughly 220,000 spoken turns in two weeks records about one turn in fifteen hitting a stall large enough to trigger a race, with recovery above 99% in about 1.2 seconds, on instrumentation that postdates the fix it measures and carries no pre-rollout baseline. Worst case, a stall now costs a bounded few seconds, roughly two to notice plus about one to recover, then the answer lands.

How do you stop two providers from both answering the same turn?

Racing creates one hard requirement. If two providers answer, the caller must hear exactly one reply. Two replies reaching the speaker, whether that is the same thing twice or two different things, is worse than the stall that caused them.

Cekura solves this with a referee, a gate that admits one winner and blocks every straggler. The gate reopens only at a clean turn boundary, so a late reply cannot slip out after an interruption has already moved the conversation on.

This is the real engineering in a racing design. Detection is a timer, and fanning a request out to several providers is a loop. Arbitrating the responses is where the correctness lives, and it depends on knowing precisely where one turn ends and the next begins, which is the same problem endpointing and turn detection solves on the input side of the pipeline.

Why does an LLM fallback need live traffic before you trust it?

Cekura treats every backup as unproven until it has carried live traffic, because a fallback is a new failure surface, and it only fails in production, under load, at the moment the primary is already down.

Providers do not share conventions, so a backup fails in ways the primary never did. Cekura hit two that testing did not surface. The first is incompatible output: a backup returns something subtly off-format from what the pipeline expects, which is as useless to the caller as silence. The second is cross-model interference: every model on a call reads and writes a shared record of the conversation, so one provider's output can break the next model that has to read it.

Neither appeared in testing. Both surfaced only once real calls ran through the fallback. Cekura load-tests a backup under concurrency before trusting it live, because a backup that rate-limits the moment you flood it buys nothing, and voice AI load testing is where that gets proven. A reliability layer is not designed once. You keep finding its edges.

When are voice agent LLM stalls most likely?

Cekura records stall rates roughly doubling at the busiest hours, on instrumentation that postdates the fix it measures and carries no pre-rollout baseline. The peak that matters is the provider's, not yours, meaning the hours when the whole internet is calling the same API.

On a hosted model your reliability is partly hostage to traffic that has nothing to do with your product. That is exactly why recovery has to be automatic and cheap. You need it most at the moment when retrying the same provider only pours more requests into the same jam. Racing to a different provider turns "everyone is slow right now" into a hiccup.

Platform choice moves the baseline as well. Cekura's benchmarks record mean response times ranging from 1.27 to 3.08 seconds across seven tested configurations. Two caveats travel with those figures: providers chose their own runtime, models and speech components, so the numbers compare complete configurations rather than isolating any single layer, and the measure is Cekura's main-agent timing rather than provider component timing. Cekura's wider account of production failure modes is in architecting resilient voice AI systems.

What does bounding the worst case cost?

Racing costs duplicate requests, and Cekura pays for them only on the turns that stall. At a stall rate of roughly one turn in fifteen, on instrumentation carrying no pre-rollout baseline, the extra spend applies to that fraction of turns and not to the rest of the traffic. That is what makes a bounded worst case affordable rather than a luxury.

Serial failover pays a different bill. It spends the full detection window plus a second request on every stalled turn, charged in caller time rather than tokens. Racing earns its spend back because the spend scales with the failure rate, not with traffic volume.

The full arithmetic, alongside the five billing layers that set a voice agent's price per minute, is worked through in Cekura's guide to voice AI agent cost and performance optimization.

What generalizes beyond voice AI?

• Bound the worst case, not the average. Users leave on the one turn that hung, not on your median. Cap every open-ended wait.
• Measure liveness, not completion. Whether it started beats whether it finished. Cekura watches the first token.
• A slow provider is a failure, not a delay. To the user a stall reads as an error, and worse, because nothing explains the wait.
• Do not add silence in the name of safety. Every retry and fallback spends real time. If it does not earn that time back, cut it.
• Race, do not switch. Run backups in parallel and take the first good answer, refereed so the user gets exactly one.
• Your fallback is a new failure surface. A backup can be slow, incompatible, or interfere with the primary, and it shows only under real load.
• You fail most when it is hardest to. Stalls cluster at the provider's peak, so recovery must route away from congestion, not retry into it.

"None of this shows when things go well. That is the point. Stalls did not stop happening. What changed is that they no longer have an open end."

Dileep Chagam, Founding Engineer at Cekura

[PLACEHOLDER, pending author sign-off. See fact base.]

FAQ

What is a silent LLM stall?

A silent LLM stall is a turn where the request is accepted and the connection stays open, but no token ever arrives. Nothing throws an error, so a conventional exception handler never fires. Cekura detects it by timing the first token instead, because liveness is the only signal that separates a stalled model from a slow one.

How long should a first-token timeout be?

Cekura uses two seconds. That threshold sits above the roughly one second a healthy first token takes, and above the 99.8% of healthy first tokens that land under two seconds, on instrumentation that postdates the fix it measures and carries no pre-rollout baseline. Set it lower and the alarm fires on good calls. Set it higher and the caller notices.

Should you retry the same LLM or fall back to another provider?

Fall back, and race rather than switch. Cekura finds same-model retries rarely recover an empty reply, and serial failover makes the caller wait on two requests instead of one. Racing several providers and taking the first to respond bounds recovery by the fastest backup, which Cekura measures at above 99% recovery in about 1.2 seconds, on instrumentation carrying no pre-rollout baseline.

What happens if two providers answer the same turn?

The caller must hear exactly one reply, so Cekura puts a referee in front of the speaker. The gate admits the first provider to start talking and blocks every straggler, then reopens only at a clean turn boundary. Without it, a late reply can surface after an interruption has already moved the conversation on.

Why test an LLM fallback before trusting it in production?

A backup only fails under real load, when the primary is already down. Cekura hit two failure modes that testing never surfaced: output subtly off-format from what the pipeline expects, and cross-model interference, where one provider's writes to the shared conversation record break the next model that reads it.

Does racing providers increase cost?

Yes, on the turns that stall. Cekura pays for duplicate requests only when the first-token watchdog fires, which at a stall rate of roughly one turn in fifteen leaves the other turns billed once. The spend scales with the failure rate rather than with traffic volume.

Building voice agents that have to survive real callers? Reliability like this is exactly what Cekura tests for. [Book a demo]