Synthflow ships four native testing methods: Phone call, Chat, Web Call and Simulation. They cover building and pre-release validation well. They do not provide independent measurement or production monitoring, which is where an external tool earns its place. Cekura tests Synthflow agents over a real telephony path and publishes independent benchmark data on the platform.
Last updated: August 2026 By Shashij Gupta
TL;DR
- Synthflow's own Simulation pairs your agent with a persona agent and scores outcomes against your success criteria, which covers pre-release validation.
- Per Cekura's benchmarks, Synthflow scores 90.4% on a single run and 81.4% when a scenario must pass three consecutive runs, a 9.0 point drop.
- Synthflow's median per-turn latency of 3.16s is the slowest of six platforms measured, while its repetition score of 4.69 out of 5 is second best.
- Native testing cannot answer how a platform compares to alternatives, because the vendor scores its own platform.
- No Cekura page currently covers Synthflow monitoring, unlike Retell, LiveKit, Pipecat and ElevenLabs.
What tools test voice AI agents built on Synthflow?
Three categories of tool apply, and they solve different problems rather than competing.
Synthflow's built-in testing covers the build loop. General-purpose evaluation frameworks handle the text layer of the language model but not the audio path, a limitation Voice Lab, one such project, states about itself: Voice Lab, an Apache-2.0 framework built specifically for voice agents, notes that "this library only supports the text part of a voice agent, i.e. testing the underlying language model and prompt." External voice testing platforms simulate full calls over telephony and monitor production, which costs real time and money per run: Vapi's voice testing documentation notes that "Each test consumes calling minutes from your account" and that "Voice tests require more time to execute compared to chat tests."
Which one you need depends less on features than on the question being asked. Building an agent is a different activity from proving it still works next month, and the tooling divides along that line rather than along vendor boundaries.
| Capability | Synthflow native | Text eval frameworks | External voice testing |
|---|---|---|---|
| Fast prompt iteration | Yes, via Chat | Yes | Not the purpose |
| Real telephony path | Yes, via Phone call | No | Yes |
| Automated scenario suites | Yes, via Simulation | Yes, text only | Yes |
| Audio quality and pacing | Partly, by listening | No | Scored |
| Cross-platform comparison | No, single platform | No | Yes |
| Production monitoring | No | No | Yes |
| Independent scoring | Vendor scores itself | Yes | Yes |
The honest reading is that Synthflow's native tooling is sufficient until you need either an outside opinion or continuous coverage after launch.
How to test a Synthflow voice agent before it ships?
Synthflow documents four methods and a recommended order, which is a reasonable default.
Per Synthflow's documentation, Chat validates conversation logic with text input and lets you "inspect turn-by-turn responses and iterate on prompts without placing calls." Phone call places a real call so you can "verify pacing, latency, and call quality." Web Call does the same from the browser. Simulation runs "automated test suites across many scenarios in parallel," pairing the agent with a persona agent and scoring outcomes against your success criteria.
Synthflow's own guidance separates the two modes clearly: manual testing "is best for fast checks while you build," while simulations "are best for pre-release validation." The suggested sequence is Chat, then Phone call or Web Call, then simulations before deployment.
The limitation is structural rather than a criticism. A platform's own simulator scores the platform it runs on, so it cannot tell you whether a different orchestrator would have handled the same scenario better.
How does Synthflow compare to other orchestration platforms?
Independent measurement exists, and it is more useful than any feature list.
Per Cekura's benchmarks, one agent was deployed unchanged across six orchestration platforms with a byte-identical, SHA-verified system prompt and four tool definitions, scored by 59 evaluators across four categories with each scenario run three times. Holding the agent constant is what makes the spread attributable to the platform.
Synthflow scores 90.4% on a single run and 81.4% on pass^3, where a scenario passes only when all three runs pass. That 9.0 point drop, obtained by subtracting the published rates, is the consistency signal. Its median per-turn latency of 3.16s is the slowest in the set and its P95 of 5.08s second slowest, while its repetition score of 4.69 out of 5 is second best.
Every figure is a comparison under one fixed test harness, not a production success rate, and the language model was held constant at gpt-4.1 at temperature 0, so the spread describes orchestration rather than model choice.
How to monitor Synthflow voice agents in production?
Production monitoring is the gap Synthflow's native tooling does not address, because its four methods all run before or outside live traffic.
What monitoring has to add is continuous scoring of real calls rather than scripted ones, alerting when a metric moves rather than when someone checks, and a path from a failed production call back into the test suite. Cekura covers the general discipline in How to Monitor AI Chat and Voice Agents in Production.
The metrics that matter on a Synthflow deployment follow from the benchmark profile above. Given the slowest median latency in the measured set, turn latency at p50 and p90 deserves a standing alert rather than a periodic review. Cekura's predefined metrics cover word error rate and character error rate for transcription, time to first token and end-of-turn detection accuracy for latency, and behavioural checks including hallucination, relevancy and response consistency, documented in A Developer's Guide to Voice AI Evaluation Metrics.
Why does independent measurement matter here?
Independent measurement matters because a vendor's own evaluator is scoring the vendor's own platform, and until recently nobody was checking whether such evaluators are accurate at all.
Cekura's answer to that is to publish the harness alongside the numbers. Cekura's published methodology records 59 automated evaluators across four categories, three runs per scenario, one agent held byte-identical across all six platforms, gpt-4.1 pinned at temperature 0, and the limitation that speech recognition was pinned on only four of the six. A buyer can check that list against any vendor's claims, including Cekura's.
The practical consequence for a Synthflow deployment is that a passing Simulation suite tells you the agent met criteria you wrote, scored by the platform under test. That is genuinely useful and it is not the same as knowing the agent works. Cekura sets out the buyer-side criteria for judging any testing platform, including its own, in How to Actually Evaluate Voice AI Testing Platforms.
Where does Cekura fit alongside Synthflow?
Cekura tests Synthflow agents from outside the platform, over a real telephony path, and keeps testing them after launch.
Cekura simulates full conversations against a live Synthflow agent, scores each turn against configurable evaluators, and monitors the same agents in production so failures return to the test set. Cekura also publishes the benchmark comparison above, which is the only figure in this post that Synthflow's own tooling cannot produce, because it requires running the identical agent on six platforms at once. Cekura maintains the same coverage for other stacks, for example testing Deepgram voice agents.
Cekura documents how these scenarios are constructed in The Complete Cekura Scenario Testing Guide. Synthflow's native testing and Cekura are complementary rather than overlapping: use Chat and Phone call while building, Simulation before release, and external testing for comparison, regression and production coverage.
The split is worth stating plainly because procurement often treats it as a choice. It is not. Removing Synthflow's native testing slows the build loop; removing external testing leaves the deployment with no independent measurement and no coverage after launch.
Frequently asked questions
Does Synthflow have built-in testing?
Yes. Synthflow provides four methods: Phone call for a real call to your number, Chat for text-based logic validation, Web Call for browser testing, and Simulation for automated suites run in parallel against a persona agent, scored on your success criteria.
How well does Synthflow perform compared to other platforms?
Per Cekura's benchmarks, Synthflow scores 90.4% on a single run and 81.4% when a scenario must pass three consecutive runs. Its median per-turn latency of 3.16s is the slowest of the six platforms measured, while its repetition score of 4.69 out of 5 is second best. These are comparisons under one fixed harness, not production success rates.
Why use an external tool if Synthflow tests agents already?
For three things its native tooling does not do: compare Synthflow against alternative platforms, score calls continuously in production, and provide measurement independent of the platform under test. Synthflow's own documentation positions its methods for building and pre-release validation.
What should you monitor on a Synthflow voice agent?
Turn latency at p50 and p90 first, given Synthflow's median of 3.16s is the slowest in the measured set, then transcription accuracy measured as WER and CER, end-of-turn detection accuracy, and hallucination rate. Monitoring differs from testing in scoring real traffic continuously rather than scripted scenarios before release.
Is Synthflow a Cekura competitor?
No. Synthflow is a voice agent orchestration platform, and Cekura is a testing and monitoring platform for the agents built on it. Cekura tests Synthflow agents, and Synthflow appears in Cekura's published benchmark as one of six platforms measured.
