multi-agent voice AI testing

Multi-agent voice AI testing verifies a phone system built from several specialized agents that hand off to each other, checking that routing, context transfer and recovery hold across the seams rather than inside one agent. Cekura runs these as scored end to end scenarios that drive a caller through every handoff on a single call.

TL;DR

• The defects live in the seams, not the agents. A suite that scores each agent alone can pass while the assembled system loses the caller's account number at the second handoff.
• Test the assembled path on one continuous call. Routing correctness, context carried across the handoff, and recovery after a wrong route are three separate assertions, and each needs its own expected outcome.
• Handoff context is a configuration choice on every major stack, not a default. LiveKit's docs name three options, pass full history, summarize it, or start fresh, while Vapi and Retell force the same decision through their own mechanisms, so the thing under test differs per deployment.
• Stage pass rates multiply. Four handoffs at 95% each complete 81% of the time, which is arithmetic rather than a measured result, and it is why per-agent scores read better than the system deserves.

What is multi-agent voice AI testing, and what does single-agent testing miss?

Multi-agent voice AI testing is the practice of exercising a voice system whose conversation is split across several agents, and asserting on the transitions between them. A triage agent qualifies the caller, hands to a booking agent, which hands to a payment agent. Each carries its own prompt, tools and often its own model.

Single-agent testing misses the transitions by construction. Score the booking agent alone and it answers well, because the harness hands it a clean context the live triage agent may not produce. The failure that reaches production is the one where routing fires on the wrong intent, or the receiving agent starts without the account number the caller already gave.

The literature outside voice points the same way. Cemri et al., "Why Do Multi-Agent LLM Systems Fail?", built a taxonomy from 1,600 or more annotated traces across seven multi-agent frameworks and sorted 14 failure modes into three categories: system design issues, inter-agent misalignment, and task verification. Those traces are coding and math tasks rather than phone calls, so read the taxonomy as a map of where to look, not as voice evidence. Turn-level scoring misses most of it, which is the argument in our single-turn testing piece.

How do you automate multi-agent voice AI testing for complex customer service workflows?

Automating multi-agent voice AI testing means writing a scenario that plays one caller across the whole workflow and asserting at each seam, rather than writing one scenario per agent. The simulated caller has to survive the handoff, because that is the event under test.

Three assertions carry most of the value. Routing: the correct receiving agent took the call, given the intent the caller actually expressed. Context: a fact stated before the handoff is still known after it, tested by having the caller refuse to repeat it. Recovery: when the route is wrong, the system returns the caller to a working path instead of looping.

Cekura scripts that caller with conditional actions, where each action pairs a trigger condition with what the testing agent then says or does. The docs describe a condition as "the trigger condition that determines when this action should be taken", and an Action Followup type that fires on the next turn after a referenced condition. That is what lets one scenario branch: send a DTMF tone into the menu, wait for the transfer, then withhold the account number and see whether the second agent already has it. The menu leg is covered further in our call transfer and IVR handoff testing guide.

How do you scale multi-agent voice AI testing across enterprise telephony platforms?

Scaling multi-agent voice AI testing across telephony platforms starts with an uncomfortable fact: the thing you are testing is not the same on each one. Handoff context is configurable, and the configuration is the variable that decides whether your context assertion means anything.

LiveKit's workflow docs are explicit that on a handoff, "Control transfers fully to a new agent. The original agent doesn't participate afterward", and that the developer chooses to pass chat_ctx, summarize, or start fresh. Vapi's Squads documentation makes the same choice explicit and warns that you may want to limit message history to reduce tokens, improve performance and prevent context poisoning. Retell's conversation flow reaches the same place through node transition conditions.

So the portable artifact is the scenario, not the assertion set. Write the caller behavior once, then declare per deployment what the receiving agent is supposed to know. Cekura connects to LiveKit, Pipecat, Vapi, Retell, ElevenLabs and SIP endpoints, and the same evaluator runs against each, which is what makes a cross-platform comparison honest. Enterprise deployments add role-based access and API key scoping on top, so runs stay attributable per team. Our orchestration tools overview covers the routing layer itself.

What performance monitoring strategies work for multi-agent voice AI testing in production?

Monitor multi-agent systems at the seam, with three signals a single-agent dashboard does not carry: handoff count per call, per-transition latency, and the rate at which a caller is routed twice. A rising double-route rate is the earliest visible sign that an upstream prompt change broke intent classification.

Repeats matter more here than in single-agent work, because handoff defects are intermittent. Cekura's published voice agent benchmarks show the size of that gap on a fixed study: across seven platform configurations, 82 scenarios and three retained repeats each, task completion ran from 87.80% to 97.56% while reliability measured as all three repeats passing ran from 30.49% to 75.61%. Those are single-agent configurations on one scenario set, not multi-agent results, and the Vapi row scores 205 calls of 246 because 41 did not connect. Read it as evidence that one green call proves very little, not as a handoff figure.

The same logic explains why per-agent scores flatter the system. Four handoffs at 95% each yield 0.95 to the fourth power, which is 81%. That is arithmetic on assumed inputs, not measurement. Re-run the suite on every prompt change, as in regression testing after prompt changes.

Which platform should you use for multi-agent voice AI testing?

Choose on whether the platform can hold one caller across the whole workflow. Most evaluation tools score a transcript after the fact. Testing handoffs needs a caller that behaves differently depending on what the agent just did, which is a live-call capability, not a scoring one. That one criterion eliminates most of the market before price enters the conversation, so apply it first and compare cost across whatever survives.

Building this yourself is reasonable and some teams should. The first suite is cheap. The cost is the second year: keeping a simulated caller convincing as models change, rebuilding the harness each time a platform alters its media handling or transfer semantics, and recalibrating judges so a score means the same thing in March and September. Price that maintenance before you decide, because it is the part that gets underestimated. Book a demo to run one of your own handoff paths.

Frequently asked questions

How does multi-agent voice AI testing pricing compare across the platform options?

Pricing models differ more than headline rates. Per-seat pricing suits teams that browse results; per-run or per-minute pricing suits suites that execute often. Multi-agent suites are long-call and repeat-heavy, so ask specifically how a vendor charges for repeats and for call minutes, since a three-repeat suite pays that multiplier on every release.

What do engineering teams actually use for multi-agent voice AI testing?

Most teams start with manual calls and a spreadsheet, then move to scripted scenarios once the agent count passes two or three. The usual stack is the orchestration platform's own logs for tracing, plus a dedicated testing platform such as Cekura for scenario execution and scoring. Purely internal harnesses are common early and tend to stall on caller simulation.

How do you fit multi-agent voice AI testing into CI?

Trigger the suite on the pull request that changes any agent prompt, tool definition or routing rule, and gate the merge on the handoff assertions rather than on aggregate score. Cekura exposes runs through its API and a GitHub Actions workflow, so a failed routing assertion blocks the merge the same way a unit test does.

What do enterprise buyers need for compliance and audit here?

Enterprise buyers need per-transition evidence, not a pass rate. That means the transcript, the metric result and the run identity retained for each handoff, plus role-based access so a run is attributable to a team. Cekura retains transcript and per-metric results per run and supports role-based access control and scoped API keys for that purpose.

Does Cekura handle multi-agent voice AI testing?

Yes. Cekura drives one simulated caller across an entire multi-agent workflow on a single call, branches that caller with conditional actions including DTMF for IVR menus, and scores routing, context retention and recovery as separate metrics. Cekura runs the same evaluator against LiveKit, Pipecat, Vapi, Retell, ElevenLabs and SIP deployments.