Function calling and tool-use testing for voice agents verifies that each spoken request becomes the right tool call, with the right arguments, at the right moment, and that the agent reports the real result. Cekura runs these tests as simulated calls against mock tools that return success, error or slow responses, then scores the tool calls themselves.
TL;DR
- Wrong argument values were the largest category of audio-induced tool-call failures in the 2026 study “From Text to Voice” (Laskar et al., TMLR), which converted the Confetti tool-calling benchmark to synthetic speech: 57.2% of those failures for Gemini-3.1-Flash-Live and 54.3% for GPT-Realtime-1.5, against 11.8% and 5.5% for wrong tool selection.
- A correct transcript does not prove a correct tool call. In Cekura’s voice agent workflow benchmark, one example call transcribed a phone number correctly and sent a different number to the tool.
- An agent can describe a tool call it never made. One example call in Cekura’s voice agent workflow benchmark narrated a tool call and continued with an invented result, so assertions must read the tool-call record, not the agent’s words.
- Mock tools make failures attributable. Cekura mock tools return fixed outputs, HTTP error codes and delays of up to 30,000 ms, so a failed test points at the agent rather than a flaky backend.
- One passing run proves little. On τ-bench, a 2024 text benchmark of function-calling agents, a gpt-4o agent succeeded on fewer than 50% of tasks, and its pass^8, the chance a task passes all eight trials, fell below 25% in the retail domain (Yao et al.).
What is function calling and tool-use testing for voice agents?
Function calling and tool-use testing for voice agents is a QA practice that checks the actions an agent takes during a call, not only its words. Cekura fetches tool call names, arguments, results and latency from VAPI, Retell, ElevenLabs, LiveKit and Pipecat test runs and attaches them to the transcript (Cekura tool call testing guide).
“In the transcript, it books the right appointment. On the phone, it talks over the caller, mishears a date of birth and goes quiet while a tool runs.” Sidhant Kabra, Co-founder & President, Cekura
A tool test answers four questions. Did the agent pick the right tool? Did the arguments match what the caller said? Did the call happen once, in the right order? Did the agent relay what the tool returned?
Speech makes the first two questions harder. In “From Text to Voice”, a paper accepted to the journal TMLR, Confetti tool-calling scores fell 1.8 to 4.8 points from text to synthetic speech for the three models compared in both modes (Laskar et al., 2026).
How do tool calls fail on voice agents?
A tool-call failure on a voice agent is a mismatch between what the caller asked for and what the backend received or the agent reported, and it takes six recurring forms. Cekura’s Tool Call Success metric flags only calls that returned an error, so most of these forms leave a call that sounds fine while the wrong thing happens in a backend nobody checked.
| Failure | What it looks like on the call | Test that catches it |
|---|---|---|
| Wrong tool | Caller asks to cancel, agent calls the order lookup | Assert the expected tool name, plus a list of tools that must not fire |
| Wrong or misheard argument | Transcript shows the right phone number, the tool receives another | Exact match on normalised argument values against the test profile |
| Narrated, never made | “Let me book that for you”, then no tool call | Match every spoken action claim to a recorded tool call |
| Result ignored or invented | Tool returns no slots, agent confirms a slot anyway | Mock a specific result and assert the agent relays it |
| Timeout and dead air | Slow API leaves the caller in silence, or the step is skipped | Mock a delay and an error status, then score silence and recovery |
| Double write | An interruption or retry fires the booking tool twice | Count calls per write tool and expect exactly one |
Cekura’s voice agent workflow benchmark publishes example calls for several modes. In the Retell configuration, second on pass³ (scenarios where all three runs passed) at 75.6%, the transcript captured a phone number correctly but a different number was sent to the tool. In an ElevenLabs call, the agent narrated a tool call and continued with an invented result. In a noisy GPT Realtime run, a long pause was followed by lost digits and a skipped tool action. Each is a single example call, not a failure rate, and providers chose their own models, speech components and settings. Hallucination detection for voice AI covers invented results.
How well do voice AI models handle tool errors and caller corrections?
Tool-error handling is a voice agent’s ability to keep a call on track when a backend call fails, and it varies widely by model. In Cekura’s speech-to-speech benchmark, 12 realtime models ran the same Pipecat agent, prompt and tools across 82 scenarios, three runs each. On the “Tool or system errors” calls the average pass rate was 88%: five models scored 100% and GPT Realtime 2.1 Mini scored 25%, with a note that it “rarely recovers from a tool error”. Calls where the caller corrects or changes their mind averaged 77%, from 96% for Grok Voice 3.0 to 38% for Gemini 3.8 Live. Recovery and restraint are separate skills: Phonic v1 passed every tool-error call, yet Cekura’s notes say it “Makes extra tool calls more often than most.” The page does not publish the scenario count per group, so read the spread as directional.
Why does tool calling break more often in voice than in text?
Voice tool calling is text tool calling in which the model must also recognise spoken argument values, and that step changes the arguments it sends. In “From Text to Voice” (Laskar et al., 2026), which converted the Confetti tool-calling benchmark to synthetic speech, wrong argument values were the largest category of cases the model got right in text and wrong in audio, for all three models analysed: 57.2% for Gemini-3.1-Flash-Live and 54.3% for GPT-Realtime-1.5, against 11.8% and 5.5% for wrong tool selection. Decision errors, a wrong high-level tool-use decision, ranked second at 25.8% and 37.4%. The authors write that models often “fail to recognize the exact argument content from speech”.
VoiceAgentBench (6,000+ synthetic spoken queries) found ASR-LLM pipelines beat end-to-end speech models, reaching up to 60.6% average parameter-filling accuracy on English (Jain et al., 2025). Cekura’s speech-to-speech benchmark points the same way: with one Pipecat agent, every saved detail matched on 94.1% of data-saving calls for the unranked cascade reference, against 59.9% to 92.0% for twelve realtime models.
How do you test function calling in a voice agent with mock tools?
Mock tools are stand-in endpoints that return a predefined response for each tool input, which makes a tool-call test deterministic and keeps it off production systems (Cekura mock tools docs). Without them, a failure could mean a broken agent or a down staging API.
Cekura auto-fetch reads tool definitions from VAPI, Retell, ElevenLabs, Bland or an agent’s own MCP server, and generates sample data from recent call transcripts. LiveKit agents need manual setup.
Give each tool three payloads:
- Success: the normal output, plus edge values such as an empty slot list.
- Error: a status code from 400 to 599, such as a 503.
- Slow: a delay_ms value up to 30,000, which exposes dead air.
Cekura returns the mock entry whose input best matches what the agent sent, so pin open-ended values such as dates with test profiles. The tradeoff: mock data drifts from the real API unless you re-fetch it after tool changes. For scenario structure, see how to write test cases for voice agents.
How do you test that a voice agent called the right tool with the right arguments?
A tool-call assertion is a rule that reads the recorded tool calls and returns pass or fail, whatever the conversation sounded like. For one scenario, write the requirement into the expected outcome, for example that the agent must call book_appointment with the requested date and time before confirming aloud. For rules that apply to every call, such as never calling transfer_to_human before identity is verified, write an LLM judge or Python metric.
Python suits exact checks on simulated runs, where the test profile supplies the expected value. Cekura’s transcript format records each tool call as a Function Call entry, so a short metric can count calls and compare digits:
import json
calls = [t for t in data["transcript_json"] if t.get("role") == "Function Call"]
books = [c for c in calls if c["data"]["name"] == "book_appointment"]
args = books[0]["data"]["arguments"] if books else {}
args = json.loads(args) if isinstance(args, str) and args.strip() else args
args = args if isinstance(args, dict) else {}
digits = lambda s: "".join(ch for ch in str(s) if ch.isdigit())
expected = digits((data.get("test_profile") or {}).get("phone", ""))
_result = len(books) == 1 and expected != "" and digits(args.get("phone", "")) == expected
_explanation = f"{len(books)} booking call(s), phone sent: {args.get('phone')}, expected: {expected or 'missing from test profile'}"
Idempotency needs its own scenario. LiveKit’s documentation notes that a user interruption does not cancel a running tool, and that an interrupted write can leave an operation partially complete with no way to roll back (LiveKit function tools docs). Barge in mid-booking and assert one write, not two. LiveKit’s docs show run_ctx.disallow_interruptions() for blocking interruptions during a write.
Which metrics should you track for voice agent tool calls?
Voice agent tool-call metrics are the numbers that show whether actions, not answers, are correct across a suite or a week of traffic. Cekura’s voice AI evaluation metrics guide defines tool success as successful required tool calls divided by all required tool calls, measured on a fixed test set so versions stay comparable. Tool Call Success is narrower: it flags errors in calls that were made, so pair it with Mock Tool Call Accuracy, which also reports expected tools that were not called.
| Metric | What it measures | How Cekura scores it |
|---|---|---|
| Tool errors | Whether any tool call returned an error | Tool Call Success, predefined, 0 credits |
| Argument accuracy | Expected tools called with the expected inputs | Mock Tool Call Accuracy, an F1-style score from 0 to 5 |
| Hallucinated calls | Spoken action claims with no matching call | Tool Call Hallucination metric, or a Python metric |
| Duplicate writes | Write tools fired more than once per request | Python metric that counts calls |
| Dead air | Mutual silence above a threshold | Detect Silence in Conversation, default 10 seconds |
| Task outcome | The caller’s goal was reached | Expected Outcome, 0 credits |
| Repeatability | Scenarios that pass on every run | pass³ across three runs |
Tool success and task outcome diverge, so track both: Cekura’s speech-to-speech benchmark notes “every tool call right, outcome still missed” for Gemini 3.8 Live Extended Thinking on plain bookings. Repeatability needs several runs per scenario, which is why τ-bench introduced pass^k and Cekura’s benchmarks report pass³. Cost differs by metric type: Cekura’s docs list Tool Call Success, Expected Outcome and Detect Silence in Conversation at 0 credits and Python metrics as free, while LLM judge metrics cost 0.2 credits per evaluation.
How do you monitor voice agent tool calls in production?
Production tool-call monitoring is the practice of scoring the recorded tool calls on every live call, where real backend errors and latency appear because nothing is mocked. Cekura ingests production calls through integrations for Retell, VAPI, ElevenLabs, LiveKit and Pipecat, or through an API for custom stacks, and runs Python and LLM judge metrics on them. Custom stacks must send function_call entries, or tool metrics have nothing to score.
Count, order and result-handling checks carry over from testing, but checks against expected values do not: Cekura’s Python metric docs list expected_outcome and test_profile for simulated runs only. Python metrics can also read provider execution logs, such as “Tool call failed: get_order timed out after 5s”.
- Keep deterministic tool checks on every call, because Python metrics consume no credits.
- Sample LLM judge metrics: a 30% sampling rate scores exactly 30 of every 100 live calls.
- Check tool results for injected text, one route for prompt injection on voice agents.
Frequently asked questions
How many times should you run each tool-call test?
Run each scenario at least three times and count it as passed only when every run passes. Results vary from run to run: on τ-bench, a 2024 text benchmark, a gpt-4o function-calling agent succeeded on fewer than 50% of tasks, and its pass^8, the chance a task passes all eight trials, fell below 25% in retail. Cekura’s workflow benchmark scores pass³ over three retained runs per scenario. Three runs triple the simulated minutes you pay for.
Can you test voice agent tool calls in text mode?
Yes for logic, no for arguments that come from speech. Chat-mode runs check tool choice, ordering and result handling quickly and cheaply, while voice runs add speech recognition and noise. Cekura’s docs describe tools that fire in Retell’s native text simulation but not in Cekura’s voice tester, caused by voice activity detection cutting the agent off before dispatch or by different model configurations. Running the same evaluator in both modes isolates the cause.
How do you test a tool timeout without touching the real backend?
Point the agent at a mock tool that waits before it answers. Cekura mock entries accept a delay_ms value from 0 to 30,000 and an error status code from 400 to 599, so one scenario can return a 503 after two seconds. Then assert three things: the agent tells the caller it is checking, it does not invent a result, and it offers a next step.
What is the difference between tool call success and task completion?
Tool call success asks whether each tool call ran without an error, while task completion asks whether the caller’s goal was reached, and either can pass while the other fails. A booking with a wrong phone number runs without error and still fails the caller. Cekura’s speech-to-speech benchmark notes “every tool call right, outcome still missed” for Gemini 3.8 Live Extended Thinking on plain bookings. A retry after a 503 can reach the goal despite one errored call. Score both.
Which tool-call tests should block a release?
Block a release on deterministic checks: the expected tool is called, the write tool fires exactly once, and argument values match the test profile. Run those on every pull request with Python metrics, which consume no credits, though each simulated voice call still bills per minute. Add an LLM judge check that an injected 503 or delay produces a recovery instead of an invented result, and run it nightly or pre-release at 0.2 credits per evaluation.
Does Cekura test function calling and tool use for voice agents?
Yes. Cekura captures tool names, arguments, results and latency from VAPI, Retell, ElevenLabs, LiveKit and Pipecat runs, mocks tools with success, error and delayed responses, and scores simulated calls with Tool Call Success, Mock Tool Call Accuracy, expected outcomes and Python metrics. Python and LLM judge metrics also run on production calls. Book a demo to see your agent’s tool calls scored.
