PolyAI ranks first of ten voice agent platforms on reliability, task completion and tool call accuracy, the three measures of whether a call actually got done.
Cekura builds testing and monitoring for voice AI agents and publishes Cekura Bench, an open set of benchmarks for voice agent platforms and the models behind them. Its voice agent workflow benchmark asks a practical question: give every platform the same job, and see which one does it reliably.
In the current release, PolyAI ranks first of ten. Its agents passed 87.8% of the benchmark's 82 caller scenarios on all three runs, 12.2 points ahead of Retell in second. They completed the caller's task on 98.8% of calls and scored 4.90 out of 5 for tool call accuracy. All three are the highest in the benchmark.
A benchmark built around real phone work
Most voice AI benchmarks score one component at a time: how accurately a model transcribes, how fast a voice starts speaking. Enterprises deploy something else: a complete agent that has to follow a long set of instructions, call business systems with the right details and get the caller to an outcome. The workflow benchmark tests that whole agent.
Every platform builds the same two agents from the same published definitions:
-
A clinic appointment line. The agent books, reschedules, cancels and looks up appointments through four tools: patient lookup, availability, booking and cancellation. It must read details back before booking, give no medical advice, and stop everything to send a caller with chest pain to 911.
-
A Medicare intake line for an insurance brokerage. The system prompt runs to about 5,000 words of compliance rules. The agent must give the required disclosure before discussing plans, get permission before collecting or sharing anything, and never recommend a plan, quote a price or promise eligibility, all of which belong to licensed agents. Its four tools run in sequence, recording permissions, saving qualification details, routing the call and writing a handoff summary, and each step has to carry forward the IDs returned by the one before.
Simulated callers then work through 82 scenarios, 59 for the clinic and 23 for Medicare. They range from a straightforward booking to the calls that break agents in production: a caller who interrupts the greeting, changes the visit reason halfway through, goes silent mid-call or coughs through a phone number; a confirmation read back over a café conversation; a booking system that returns an error; a prompt injection hidden in a name field; a caller claiming developer access and asking for hidden instructions; and a family member calling on behalf of a parent.
How every call is scored
How the Cekura Bench workflow benchmark works, from provider configuration to pass³.
Each platform chose the configuration it wanted tested, with its own models, speech components and settings. That makes the result a measure of the platform as a customer would deploy it, not of a single model in isolation. Every platform received the same brief: system prompts, tool definitions, mock data and test data, with transfers to other agents as the only restriction.
Every call is scored by the same evaluator suite. It checks whether the caller reached the expected outcome, whether each tool was called with exactly the right arguments, whether the call stayed connected and responsive, how the agent handled interruptions, how quickly it replied and how it sounded.
The headline measure is pass³: the share of scenarios an agent passed on all three of its runs. Callers don't get a best of three. An agent that passes 90% of its calls, with misses landing at random, gets a given scenario right three times in a row only about 73% of the time, and at enterprise call volumes that gap reaches real callers every day. pass³ rewards the agent that does the job the same way every time.
Calls that fail to connect or produce no transcript stay in the results. The agent definitions, tools and mock data are open source, every call recording is public, and vendors can flag errors before publication but cannot edit or veto their results.
Why PolyAI's result stands out
Share of the 82 scenarios each platform passed on all three runs. Higher is better.
The margin. Most of the field sits between 56% and 76% on pass³. PolyAI's 12.2-point lead over second place is wider than the gap between second and sixth.
The consistency. PolyAI passed 94.7% of its individual calls, and its pass³ is only 6.9 points lower. That is the smallest drop of any platform; for the other nine it ranges from 11 to 23 points. When PolyAI's agents handled a scenario correctly, they almost always handled it correctly again.
The job done. PolyAI completed the caller's task on 98.8% of its 246 calls, the only platform above 98%. It also scored highest for tool call accuracy, at 4.90 out of 5. That matters because tool calls are where this benchmark separates platforms. Many of the failures it catches have nothing to do with understanding the caller. They are details: a phone number heard correctly and then sent to the booking tool wrong, a consent ID collected and then left out of the handoff, an agent that describes a tool call and carries on with a result it made up. Calling the right tool with the right arguments, call after call, is what turns an understood request into a finished one.
PolyAI also scored 4.99 out of 5 for handling interruptions, second in the field, and 99.2% of its calls were free of connection or platform issues.
PolyAI against the other nine platforms on reliability, task completion and tool call accuracy. Higher is better.
Built for enterprise customer service
PolyAI has built voice agents for enterprise customer service since 2017. Its founders are alumni of the University of Cambridge's Dialogue Systems Group, and its agents answer calls for brands such as Marriott, Gordon Ramsay Restaurants, PG&E and UniCredit.
PolyAI describes its product as dialog agents that finish what customers start, which is what this benchmark measures. It builds both the platform and the models underneath it. Its platform runs modular pipelines alongside Dialog-RSN-1, its newest audio-native reasoning model, which listens to the raw call audio and uses the conversation and the system prompt as context for what it hears.
How PolyAI's agents were built
PolyAI built both benchmark agents on its own platform and ran them as a customer would deploy them.
The platform runs two kinds of voice agents. A modular pipeline chains speech recognition tuned for phone audio, a language model and a voice, so each part can be chosen and tuned for the calls it handles. Dialog-RSN-1, PolyAI's audio-native model, handles turn-taking, speech recognition, function calling and the reply in one model that listens to the raw call audio. Each deployment runs on the architecture that fits its calls. What decides whether a call gets done sits in the platform around the model: how the agent follows a long prompt, calls tools and stays inside its rules.
That is where this benchmark separates platforms. The Medicare line chains four tools, and each needs the IDs returned by the one before, so one dropped consent ID fails the handoff. It also runs on about 5,000 words of compliance rules, including scenarios built to pull the agent off them: a prompt injection hidden in a name field and a caller claiming developer access.
PolyAI tests every agent before it goes live, whichever architecture it runs on. Both benchmark agents went through internal testing and PolyAI's own evaluation benchmarks, which score agents on live-call conditions such as real audio, noise and tool use. PolyAI also works with third parties like Cekura, whose independent benchmarks check the whole deployed agent, tool calls included.
“A voice agent isn't judged on whether it understands the caller. It's judged on whether the job got done, the same way every time,” said Eddy Su, co-founder and SVP, Engineering at PolyAI. “That consistency is what this benchmark measures, and it's what we build every PolyAI agent to deliver, on a modular pipeline or on Dialog-RSN-1.”
See every call
The full results for all ten platforms are on the voice agent workflow leaderboard, with head-to-head comparisons such as PolyAI vs Retell and every recording from PolyAI's run.
Learn more about PolyAI's platform at poly.ai, or try Dialog-RSN-1, its audio-native model, on your own calls at poly.ai/dialog-rsn-1. To run the same 82 scenarios against your own agent, start with the benchmark runner on GitHub or talk to Cekura.