Voicemail detection testing for voice AI checks whether an outbound agent correctly classifies what picked up the call, then takes the right branch. Cekura places real calls to your agent, plays voicemail greetings, IVR menus and screeners at it, and scores each branch, so a misclassification fails a test run instead of a campaign.
TL;DR
- Voicemail detection testing asserts two things separately: the verdict your platform returned, and the branch your agent took after it. Most suites check only the second and miss the cause.
- The scenarios that catch real defects are the short greeting, the greeting with no beep, the IVR menu, the silent answer and the phone-side call screener.
- Detection settings are tunable, so a test suite has to pin them. Twilio's answering machine detection ships four tuning parameters with published defaults, and Vapi exposes a detection provider choice plus a beep wait window.
- Cekura plays the callee side: a
<voicemail>or<ivr>block in a test scenario makes your agent face a recorded greeting or a keypad menu over real telephony, with DTMF returned to the test agent. - Score with repeats and keep failed connections in your reliability denominator. A single passing call proves very little about a classifier that fires in the first five seconds.
What is voicemail detection testing for voice AI?
Voicemail detection testing for voice AI is a test discipline that verifies answer classification and the branch that follows it. Answer classification is the moment, usually three to five seconds into an outbound call, when the platform decides whether a person, a voicemail greeting, an IVR menu, hold music or a carrier message picked up. Everything the agent does next depends on that verdict being right.
The discipline is narrower than testing the conversation. A conversational test suite checks whether the agent booked the appointment. A voicemail detection suite checks whether the agent should have been talking at all. Those fail independently, and a green conversation suite tells you nothing about the classifier, because the classifier already decided correctly in every scenario you wrote.
Two failure directions cost different amounts. A false machine verdict means the agent hangs up on a real person or talks over them, which burns the lead and can generate a complaint. A false human verdict means the agent delivers its opening line into a greeting and then holds a conversation with silence until a timeout fires. Our guide to AI outbound calling covers where this sits in the wider call path.
Why does voicemail detection fail on outbound AI calls?
Voicemail detection fails because the two classes sound alike. As the arXiv preprint Real-Time Voicemail Detection in Telephony Audio Using Temporal Speech Activity Features puts it, detection "cannot rely on spectral content differences: voicemail greetings are human speech, recorded by the same person who would answer a live call." The same paper notes that at 8 kHz, frequencies above 4 kHz are absent, which limits the utility of high-frequency spectral features.
Vendors say the same of their own engines. Twilio's answering machine detection documentation states that "Since not all humans and not all voicemail greetings follow similar patterns in answering calls, it's possible that AMD will not always return the right answer." It gives the concrete example that the engine "may, for example, interpret a very short two-second voicemail greeting as a human picking up."
That preprint reports a system reaching 96.1 percent combined accuracy on 764 telephony recordings, and a 0.3 percent false positive rate over 77,000 production calls. Read it as one vendor's self-reported result on English-language telephony, not an industry figure. What transfers is the shape of the errors, not the rate. Our voicemail detection guide covers the detection methods and how they trade speed against accuracy.
Which scenarios belong in a voicemail detection test suite?
A voicemail detection test suite is a set of scenarios where something other than a cooperative human answers. Six are worth writing deliberately, and each maps onto a documented failure mode rather than a guess. A short greeting under three seconds, which Twilio names explicitly and which the arXiv preprint reports as its single false negative on the expert-labelled set. A long carrier greeting, where the agent must wait for the beep rather than talk over it. A greeting with no beep, common on personal and third-party voicemail. An IVR menu that expects keypad input. A silent answer, where the line opens and nobody speaks. A phone-side call screener that answers on the callee's behalf, which the arXiv authors call "indistinguishable from live humans using temporal features alone."
Cekura runs these from the callee side. A <voicemail> or <ivr> block inside a test scenario makes Cekura's test agent play a recorded-style greeting or a keypad menu at your agent, and the IVR and voicemail documentation requires the Receive DTMF toggle so the test agent picks up tones from your side. Cekura also holds a multi-part menu open with an interruption-suppressing block, so a menu playing in several clips is not cut short.
How do you automate voicemail detection testing for enterprise voice AI agents?
Automating voicemail detection testing means running the scenario set on a trigger rather than by hand, and pinning the settings that change the verdict. Detection is configuration-sensitive, so a suite that does not record the configuration is not reproducible.
Pin the numbers first. Twilio publishes four tuning parameters with defaults: a 30 second detection timeout, a 2400 millisecond speech threshold, a 1200 millisecond speech end threshold and a 5000 millisecond silence timeout. Vapi's voicemail detection documentation instead exposes a provider choice, a minimum check interval of 2.5 seconds, and a beep wait window defaulting to 30 seconds, warning that under 15 to 20 seconds "may cause your voicemail message to be cut off." Test at your configured values, then test one step either side of them.
Then run the set on every release, which means picking a tool that can trigger a run, not only display results. Cekura triggers outbound runs from the dashboard or the API against your provider, and the outbound dial window closes 300 seconds after it opens by default, returning a timeout status if the dial never arrives. Schedule the same set on a cadence as well, which our automated recurring tests guide covers, because carrier behaviour drifts without any change on your side.
Which platform should you use for voicemail detection testing?
The choice is really between four approaches, and the criteria that separate them are coverage, setup cost, whether the check survives into continuous integration, and what each one cannot see. Cekura sits in the fourth row. Price follows the approach rather than the vendor, because every row that dials a real number costs call minutes, and the two rows that cost nothing in minutes are the two that prove nothing before the campaign opens.
| Approach | What it covers | Setup effort | Runs in CI | Blind spot |
|---|---|---|---|---|
| Dial your own handset by hand | One greeting, one carrier, one moment | Minutes | No | Short greetings, IVR menus, screeners, drift |
| Read production dispositions after a campaign | Real traffic, after the fact | Minutes | No | Nothing is proven before the dial |
| Build a recorded-greeting harness in-house | The greetings you recorded | Weeks of engineering plus telephony spend | Yes, while someone maintains it | Carrier variation, IVR menus, upkeep after prompt changes |
| Cekura outbound evaluators with voicemail and IVR scenarios | Greeting length, missing beep, IVR menus, DTMF, silence, screeners, scored per run | One agent integration | Yes, on a schedule or a pipeline trigger | Carrier-level SIP signalling, which application-layer platforms generally cannot see |
Cekura is the tool that puts a controlled greeting on the callee side of a real call, which is the only way to assert the verdict rather than infer it from an outcome. Cekura scores each scenario against criteria you define, so the branch is a pass or a fail rather than a transcript somebody reads. Enterprise buyers with compliance or audit requirements need more than a verdict, since an auditor asks which scenarios ran and what each one scored. Cekura leaves that record behind on every scheduled run, as a per-scenario pass or fail. Writing the assertions is the work that remains, and our guide to writing test cases for voice agents covers how to phrase them so they fail for one reason.
Should you build voicemail detection testing in-house or buy it?
Building it in-house is feasible and the cost is mostly upkeep. You need outbound telephony numbers, a set of recorded greetings covering length and beep variation, an IVR simulator that returns DTMF, a scheduler, and a scoring layer that decides pass or fail without a person listening. The recordings are the easy part. The harness that dials your agent, plays the right greeting at the right moment and captures what the agent did is where the weeks go.
Two ongoing costs decide it. Carrier and handset behaviour changes underneath you, so a fixed recording set decays. And every prompt change reopens the question of whether the agent still handles the branch, which means the suite has to run on a trigger rather than when somebody remembers.
Buy it when voicemail is a meaningful share of your dial volume and the suite has to run on every release. Build it when you have one narrow case and no plan to extend it. Cekura's published benchmark method is worth copying either way: 7 configurations, 82 scenarios, three retained repeats, and no-connect calls retained in its infrastructure reliability denominator. That last rule is what stops a voicemail suite from flattering itself.
Frequently asked questions
What do engineering teams actually use for voicemail detection testing for voice AI?
Teams typically combine two things: the detection setting their voice platform already ships, and a test suite that dials the agent with controlled greetings. Cekura covers the second, running voicemail and IVR scenarios over real telephony and scoring the branch. Teams without a suite usually rely on production dispositions, which report failures after the campaign rather than before it.
How accurate is voicemail detection on outbound AI calls?
It depends on the method and it is not solved. Twilio states plainly that its engine "will not always return the right answer." The arXiv preprint cited above reports 96.1 percent combined accuracy on 764 recordings and a 0.3 percent false positive rate across 77,000 production calls, which is one vendor's self-reported result on English-language telephony rather than an industry benchmark. Measure your own rate.
Can a test catch an AI call screener answering instead of a person?
Yes, and it is worth a dedicated scenario. Phone-side screeners answer interactively and sound like a person, so temporal and beep-based detection both read them as human. A test scenario that plays a screener asking why you are calling shows you which branch your agent takes, which is the decision you need to make deliberately rather than discover in production.
How much does voicemail detection testing cost?
Budget the scenario count times repeats times minutes per call, because every scenario is a real outbound call. Cekura's published usage-based pricing is a per-seat base, the first user free and each additional user $30 a month, plus credits metered separately at 5 credits per voice testing minute. Building in-house replaces that with engineering time plus your own telephony spend.
Does Cekura handle voicemail detection testing for voice AI?
Yes. Cekura places outbound calls to your agent and plays the callee side, using a <voicemail> block for a recorded greeting or an <ivr> block for a keypad menu, with DTMF returned to the test agent. Cekura scores each scenario against your criteria, runs on a schedule or a pipeline trigger, and works with Vapi, Retell, ElevenLabs, Bland, LiveKit and custom providers.
