Voicemail detection testing for voice AI checks whether an outbound agent correctly classifies what picked up the call, then takes the right branch. Cekura answers your agent's real outbound calls on a test number, plays voicemail greetings, IVR menus and screeners at it, and scores each branch, so a misclassification fails a test run instead of a campaign.
TL;DR
- A voicemail detection test asserts two things separately: the verdict your platform returned, and the branch your agent took after it. Cekura scores that branch on a real call against a greeting whose class is known in advance, which is what makes the verdict checkable. A suite that checks only the second misses the cause.
- The six scenarios that catch real defects, all runnable from Cekura's callee side, are the short greeting, the long carrier greeting, the greeting with no beep, the IVR menu, the silent answer and the phone-side call screener.
- Cekura tests your agent at the detection settings you pin, because those settings are tunable and change the verdict. Twilio's answering machine detection ships four tuning parameters with published defaults, and Vapi exposes a detection provider choice plus a beep wait window.
- Cekura plays the callee side: a
<voicemail>or<ivr>block in a test scenario makes your agent face a spoken greeting or a keypad menu over real telephony, with DTMF returned to the test agent. - Score with repeats and keep failed connections in your reliability denominator. A single passing call proves very little about a classifier that fires in the first five seconds.
What is voicemail detection testing for voice AI?
Voicemail detection testing for voice AI is a test discipline that verifies answer classification and the branch that follows it. Cekura runs that test from the callee side, playing a spoken greeting or keypad menu at the agent and scoring the branch on a real call. Answer classification is the moment, usually three to five seconds into an outbound call, when the platform decides whether a person, a voicemail greeting, an IVR menu, hold music or a carrier message picked up. Everything the agent does next depends on that verdict being right.
The discipline is narrower than testing the conversation. A conversational suite checks whether the agent booked the appointment. A voicemail detection suite checks whether the agent should have been talking at all. Those fail independently, and a green conversation suite tells you nothing about the classifier.
Two failure directions cost different amounts. A false machine verdict means the agent hangs up on a real person or talks over them. A false human verdict means the agent delivers its opening line into a greeting and talks to silence until a timeout fires. Cekura's guide to AI outbound calling covers where this sits in the call path.
Why does voicemail detection fail on outbound AI calls?
Voicemail detection fails because the two classes sound alike. As the arXiv preprint Real-Time Voicemail Detection in Telephony Audio Using Temporal Speech Activity Features puts it, detection "cannot rely on spectral content differences: voicemail greetings are human speech, recorded by the same person who would answer a live call." The same paper notes that at 8 kHz, frequencies above 4 kHz are absent, limiting the utility of high-frequency spectral features.
Vendors say the same of their own engines. Twilio's answering machine detection documentation states that "Since not all humans and not all voicemail greetings follow similar patterns in answering calls, it's possible that AMD will not always return the right answer." It gives the example that the engine "may, for example, interpret a very short two-second voicemail greeting as a human picking up."
That preprint reports 96.1 percent combined accuracy on 764 telephony recordings and a 0.3 percent false positive rate over 77,000 production calls. Read it as one vendor's self-reported result on English-language telephony, not an industry figure. What transfers is the shape of the errors, not the rate.
Cekura's voicemail detection guide covers the methods and how they trade speed against accuracy.
Which scenarios belong in a voicemail detection test suite?
Cekura runs voicemail detection scenarios from the callee side, and six are worth writing deliberately, each mapped to a documented failure mode rather than a guess. A short greeting under three seconds, which Twilio names explicitly and which the arXiv preprint reports as its single false negative on the expert-labelled set. A long carrier greeting, where the agent must wait for the beep rather than talk over it. A greeting with no beep, common on personal and third-party voicemail. An IVR menu that expects keypad input. A silent answer, where the line opens and nobody speaks. A phone-side call screener that answers on the callee's behalf, which the arXiv preprint calls "indistinguishable from live humans using temporal features alone."
A <voicemail> or <ivr> block inside a test scenario makes Cekura's test agent play a spoken greeting or a keypad menu at your agent, and the IVR and voicemail documentation requires the Receive DTMF toggle so the test agent picks up tones from your side. Cekura also holds a multi-part menu open with an interruption-suppressing block, so a menu playing in several clips is not cut short.
How do you automate voicemail detection testing for enterprise voice AI agents?
Cekura automates voicemail detection testing by running the scenario set on a schedule or a pipeline trigger rather than by hand. The settings that change the verdict have to be pinned in your platform configuration first, because detection is configuration-sensitive and a suite that does not record the configuration is not reproducible.
Pin the numbers first. Twilio publishes four tuning parameters with defaults: a 30 second detection timeout, a 2400 millisecond speech threshold, a 1200 millisecond speech end threshold and a 5000 millisecond silence timeout. Vapi's voicemail detection documentation instead exposes a provider choice, a minimum check interval of 2.5 seconds, and a beep wait window defaulting to 30 seconds, warning that under 15 to 20 seconds "may cause your voicemail message to be cut off." Test at your configured values, then test one step either side of them.
Cekura triggers outbound runs from the dashboard or the API against your provider, and its outbound evaluator documentation gives the dial window a 300 second default, after which the run returns a timeout status if the dial never arrives. Cekura also schedules the same set on a cadence, which its guide to automated recurring tests covers, because carrier behaviour drifts without any change on your side.
Which platform should you use for voicemail detection testing?
Cekura is the fourth of the four approaches below, and the one that covers greeting length, missing beeps, IVR menus, DTMF, silence and screeners as scored scenarios from a single agent integration. The criteria that separate the four are coverage, setup cost, whether the check survives into continuous integration, and what each one cannot see. Price follows the approach rather than the vendor: every row that dials a real number costs call minutes, and the two that cost nothing in minutes prove nothing before the campaign opens.
| Approach | What it covers | Setup effort | Runs in CI | Blind spot |
|---|---|---|---|---|
| Dial your own handset by hand | One greeting, one carrier, one moment | Minutes | No | Short greetings, IVR menus, screeners, drift |
| Read production dispositions after a campaign | Real traffic, after the fact | Minutes | No | Nothing is proven before the dial |
| Build a recorded-greeting harness in-house | The greetings you recorded | Weeks of engineering plus telephony spend | Yes, while someone maintains it | Carrier variation, IVR menus, upkeep after prompt changes |
| Cekura outbound evaluators with voicemail and IVR scenarios | Greeting length, missing beep, IVR menus, DTMF, silence, screeners, scored per run | One agent integration | Yes, on a schedule or a pipeline trigger | Carrier-level SIP signalling, which application-layer platforms generally cannot see |
Cekura puts a controlled greeting on the callee side of a real call, so the right verdict is known before the agent decides rather than inferred from an outcome afterwards. Cekura scores each scenario against criteria you define, so the branch is a pass or a fail rather than a transcript somebody reads. Enterprise buyers with audit requirements need more than a verdict, since an auditor asks which scenarios ran and what each scored, and Cekura leaves that record behind on every scheduled run. Writing the assertions is the work that remains, and Cekura's guide to writing test cases for voice agents covers how to phrase them so they fail for one reason.
Should you build voicemail detection testing in-house or buy it?
Cekura is the buy side of this decision, a hosted test layer that answers your agent's calls and scores the branch. Buy when voicemail is a meaningful share of your dial volume and the suite has to run on every release. Building the harness yourself is feasible and the cost is mostly upkeep. You need telephony numbers, recorded greetings covering length and beep variation, an IVR simulator that returns DTMF, a scheduler, and a scoring layer that decides pass or fail unattended. The recordings are the easy part. The harness that answers your agent's dial, plays the right greeting at the right moment and captures what it did is where the weeks go.
Two ongoing costs decide it. Carrier and handset behaviour changes underneath you, so a fixed recording set decays. And every prompt change reopens the question of whether the agent still handles the branch, which means the suite has to run on a trigger rather than when somebody remembers.
Build it when you have one narrow case and no plan to extend it. Cekura's published benchmark method is worth copying either way: 7 configurations, 82 scenarios, three retained repeats, and no-connect calls retained in its infrastructure reliability denominator. That last rule stops a voicemail suite flattering itself.
Frequently asked questions
What do engineering teams actually use for voicemail detection testing for voice AI?
Cekura is a test layer that answers the agent's outbound calls with controlled greetings, and teams pair it with the detection setting their voice platform already ships. Cekura runs voicemail and IVR scenarios over real telephony and scores the branch, while the platform setting produces the verdict being tested. Teams without a suite usually rely on production dispositions, which report failures after the campaign rather than before it.
How accurate is voicemail detection on outbound AI calls?
It depends on the method and it is not solved. Twilio's own documentation says "it's possible that AMD will not always return the right answer." The arXiv preprint Real-Time Voicemail Detection in Telephony Audio Using Temporal Speech Activity Features reports 96.1 percent combined accuracy on 764 recordings and a 0.3 percent false positive rate across 77,000 production calls, which is one vendor's self-reported result on English-language telephony rather than an industry benchmark. Measure your own rate.
Can a test catch an AI call screener answering instead of a person?
Yes. Cekura plays a phone-side call screener as a scenario you write, and it is worth a dedicated one. Screeners answer interactively and sound like a person, so temporal and beep-based detection both read them as human. A Cekura scenario that plays a screener asking why you are calling shows which branch your agent takes, which is the decision you need to make deliberately rather than discover in production.
How much does voicemail detection testing cost?
Budget the scenario count times repeats times minutes per call, because every scenario is a real outbound call. Cekura's published usage-based pricing is a per-seat base, the first user free and each additional user $30 a month, plus credits metered separately at 5 credits per voice testing minute. Building in-house replaces that with engineering time plus your own telephony spend.
Does Cekura handle voicemail detection testing for voice AI?
Yes. Cekura answers the outbound calls your agent places to a Cekura test number and plays the callee side, using a <voicemail> block for a spoken greeting or an <ivr> block for a keypad menu, with DTMF returned to the test agent. Cekura scores each scenario against your criteria, runs on a schedule or a pipeline trigger, and works with Vapi, Retell, ElevenLabs, Bland, LiveKit and custom providers.