Voice AI testing for hospitality means simulating the calls a property actually receives: multilingual guests, peak check-in concurrency, and property-specific detail the agent must retrieve correctly rather than invent. Cekura runs these as scored simulations across 32 languages and 31 background environments, then applies the same evaluators to live production calls.
Last updated: August 2026
TL;DR
- Hospitality fails on language: the agent detects a guest's language once and then drifts back to English mid-call.
- Retail and ecommerce fail on state: the agent states stock or order status confidently from data that has already changed.
- Real estate fails on entity capture: a misheard street name or unit number sends the whole call to the wrong property.
- Restaurants fail on acoustics: modifiers attach to the wrong item once the noise floor rises.
- Per Cekura's benchmarks, every one of six voice platforms scored lower when the same evaluator had to pass three consecutive runs instead of one, with the largest drop 11.8 points. These are platform defaults on one unchanged agent, not tuned per-vertical configurations.
Why the same voice agent passes in one vertical and fails in another
A vertical is not a branding layer on a voice agent. It is a different distribution of inputs: who calls, what they say, how noisy the line is, and which system of record the agent has to read before it answers. An agent that handles a quiet inbound line with English-speaking callers and a static FAQ will hold up under exactly those conditions and nowhere else.
This is why a single passing test call is not evidence. Per Cekura's benchmarks, which ran 59 evaluators across four categories with three runs each against one agent deployed unchanged on six platforms, single-run pass rates ranged from 88.1% to 98.9%. Requiring the same evaluator to pass all three consecutive runs dropped every platform, to a range of 76.3% to 96.6%. The benchmarks page defines the two measures directly: "pass^1 = one run meets the success rubric; pass^3 = all three runs do; the gap is the consistency signal."
Those figures are platform defaults on an untuned agent, and the dataset is one agent, a healthcare appointment-scheduling assistant, with no industry or vertical breakdown. It measures consistency, not suitability for hotels or restaurants. The vertical work is yours to do, and the sections below set out what each one actually demands.
What each vertical forces you to test
The table below maps five verticals against the dimension that breaks them first. It is not a ranking of platforms. It is a checklist of what your test suite has to contain before the agent takes a real call in that vertical.
| Vertical | Dominant failure mode | What the caller does that triggers it | What you must assert | Concurrency shape |
|---|---|---|---|---|
| Hospitality | Language persistence | Opens in one language, code-switches mid-sentence, uses a non-native accent | Agent holds the guest's language across every turn, including after a tool call, and every rate, availability, or folio answer traces to a property management system lookup that fired on the call | Sharp spikes at check-in and check-out windows |
| Retail and ecommerce | Stale state asserted as fact | Asks whether an item is in stock or where an order is | Every availability or status claim traces to a live lookup, and no upsell is offered on an unavailable item | Seasonal peaks lasting weeks, not minutes |
| Real estate | Entity capture on proper nouns | Says a street name, unit number, or development name the model has rarely seen | Captured address and listing ID match the record before any detail is quoted | Bursts following a listing going live or an ad running |
| Restaurants | Acoustic degradation | Orders fast, stacks modifiers, changes the order mid-sentence, with engine and kitchen noise behind | Each item, quantity, and modifier attaches to the correct line before the order is written | Hard lunch and dinner peaks, daily |
| Fintech and healthcare | Compliance and disclosure | Attempts an action that requires verification or a mandatory disclosure | Covered separately in voice bot testing for fintech and conversational AI in healthcare | Steady, regulated volumes |
The last row is there to mark the boundary. Regulated verticals fail on disclosure and verification, which is a different discipline from the operational failures above, and each is covered on its own page.
Voice AI testing for hospitality
Voice AI testing for hospitality is the practice of validating a property's voice agent against the guest mix that property actually serves, not against a generic English test set. The dominant failure is language persistence. An agent detects a guest's language on the opening turn, answers correctly once, and then reverts to English after a tool call or a barge-in, usually because language state is not carried through the turn that follows retrieval.
Test it by holding the language variable and moving everything else. Cekura's multilingual testing runs 32 languages with regional accent layering and a dedicated mode for code-switching inside a single call, so a simulated guest can open in Spanish, switch to English for a room number, and switch back. The assertion is not that the agent understood. It is that the agent replied in the guest's language on every turn, including the turn immediately after a lookup.
The second assertion is the property record. Rate, availability, folio balance, and loyalty status live in the property management system, not in the prompt, so every one of those answers has to trace to a lookup that fired on the call. Test the agent on a rate that changed between two calls and assert it quotes the second one.
The third hospitality dimension is concurrency shape. Property call volume is not flat: it spikes hard in the check-in and check-out windows, then falls away. An agent that answers quickly on a quiet overnight line is not the same agent under a check-in rush, so load testing has to reproduce your own spike rather than the daily average.
Voice AI testing for retail and ecommerce
Voice AI testing for retail and ecommerce is the practice of proving that every factual claim the agent makes about stock, pricing, or order status came from a live lookup at the moment it spoke. The dominant failure is stale state asserted as fact. The agent says an item is available, or an order shipped, because that was true when the context was assembled, and the caller acts on a claim that is no longer correct.
The assertion that catches this is a tool-call assertion, not a transcript check. A transcript reading "yes, we have that in stock" looks correct in isolation and is worthless as evidence. What you test is whether the inventory lookup fired before the claim, and whether the spoken answer matches what it returned. Cekura asserts on tool calls and their ordering, which is how a confident, fluent, wrong answer gets caught. Grounding answers in retrieval rather than prompt text is covered in knowledge base connectors and agentic retrieval.
Upsell scripting is the second retail dimension, and it interacts with the first. An attach offer is a scripted behavior with a hard precondition: the item offered has to be available. Test that the agent makes the offer when it should and never attaches it to something the lookup just returned as out of stock.
Voice AI testing for real estate
Voice AI testing for real estate is the practice of validating that the agent captured the right property before it says anything about that property. The dominant failure is entity capture on proper nouns. Street names, development names, and unit numbers are exactly the tokens speech recognition handles worst, because they appear rarely in training data and often have no phonetic neighbours the model prefers.
The research is direct about this. In their 2024 paper Retrieval Augmented Correction of Named Entity Speech Recognition Errors, Ernest Pusateri and co-authors write: "In recent years, end-to-end automatic speech recognition (ASR) systems have proven themselves remarkably accurate and performant, but these systems still have a significant error rate for entity names which appear infrequently in their training data." A property portfolio is a list of exactly such names.
Two mitigations are worth testing against. Deepgram's keyterm prompting boosts recognition of product, company, and domain-specific names within a documented budget of 500 tokens per request, with a recommendation to focus on the most important 20 to 50 terms. A live listing set exceeds that, so the boosted terms are a rotating selection of active inventory, and that selection is itself worth a test. On the output side, ElevenLabs documents pronunciation dictionaries and alias tags for controlling how specific names and acronyms are spoken. Test both directions, and score at entity level rather than on the whole utterance, as in voice agent accuracy testing.
Voice agent QA for restaurants
Voice agent QA for restaurants is the practice of testing an ordering agent under the noise floor it will actually run at, with orders complex enough to break modifier attachment. The dominant failure is not misunderstanding the item. It is attaching the right modifier to the wrong line once audio quality drops, so the ticket is plausible, complete, and wrong.
Deepgram states the acoustic problem plainly: "Drive-thru lanes are acoustically challenging environments filled with engines, wind, and overlapping speech, so training models on quiet call-center recordings will set deployments up to fail." The same holds for a phone line next to a service pass. Cekura's simulation layer carries the 31 background environments documented in its multilingual testing, plus configurable interruption levels, so the noise floor and the barge-in behavior are test variables rather than production surprises. Accent and dialect testing covers the speaker-side variation that compounds it.
Text-level tests do not reach this failure. LiveKit's testing documentation separates its own text-based framework, which returns deterministic results, from tooling that exercises "End-to-end behavior through the full audio pipeline." Modifier attachment under noise lives entirely in the second category, which is why a passing unit-test suite and a broken drive-thru coexist comfortably.
Academic work quantifies why the ticket looks right. In Identifying and Calibrating Overconfidence in Noisy Speech Recognition, Mingyue Huo and co-authors report that Whisper does not only lose accuracy in additive noise, it becomes overconfident, with 10 to 20% of tokens wrongly predicted at confidence above 0.7 at low signal-to-noise ratios. Those figures come from the R-SPIN dataset in the minus 18 to minus 5 dB range, which is more severe than a service line, so read them as the direction the failure takes rather than a drive-thru measurement. A confidence score is not an assertion, which is why the modifier still has to be checked against the line it attached to.
Order the restaurant suite like this:
- Set the noise floor to the environment the line actually runs in, not a quiet room.
- Build orders that stack at least three modifiers across two or more items.
- Change one item mid-sentence, after the agent has begun confirming.
- Assert per line: item, quantity, and modifier attach to the correct line.
- Re-run the same scenario three times and gate on all three passing, not one.
How Cekura tests vertical-specific voice agents
Cekura tests a vertical by turning its failure mode into a scored simulation and then running that simulation at volume. A single scenario expands combinatorially: Cekura's multilingual testing documentation works through 5 languages by 3 emotions by 3 speeds by 4 interruption levels by 5 background environments, which is 900 unique combinations from one scenario, and 18,000 executions across 20 scenarios. For a vertical, that is the difference between testing a hotel agent and testing the guest mix.
Cekura asserts at three levels rather than one: it scores the transcript, it asserts on tool calls and the order they fired in, and it applies the same evaluators to production calls that ran in pre-release testing, so a regression that only appears under real traffic still gets caught. Those assertions run natively against Vapi, Retell, LiveKit, Pipecat, and ElevenLabs, so the vertical suite moves with you if the underlying platform changes.
The practical sequence is to write the vertical's dominant failure as an assertion first, expand it across the personality and environment dimensions that vertical actually sees, and gate releases on the consistency measure rather than a single pass. Adjacent operational coverage sits in AI answering service, AI virtual receptionist technology, and AI appointment booking tools.
Frequently asked questions
What is voice AI testing for hospitality?
Voice AI testing for hospitality is the validation of a property's voice agent against its real guest mix, with language persistence as the primary assertion. The agent must hold the guest's language across every turn, including after tool calls, and must sustain quality through check-in and check-out concurrency spikes rather than average daily load. Cekura runs this as scored simulations across 32 languages with code-switching inside a single call.
How is voice AI testing for retail and ecommerce different?
Retail and ecommerce testing centres on state accuracy rather than language. Every claim about stock, price, or order status must trace to a live lookup that fired before the agent spoke. The test is a tool-call assertion, because a transcript saying an item is in stock reads as correct whether or not the lookup ever ran.
What does voice AI testing for real estate need to cover?
Real estate testing centres on entity capture. Street names, unit numbers, and development names are rare tokens that speech recognition handles poorly, and a single capture error redirects the entire call to the wrong property. Assert that the captured address and listing identifier match the record before any property detail is quoted.
What does voice agent QA for restaurants require?
Restaurant QA requires testing under a realistic noise floor with orders complex enough to stress modifier attachment. The assertion is per line item: each item, quantity, and modifier must attach to the correct line before the order is written. Text-level tests cannot reach this failure because it originates in the audio pipeline. Cekura carries 31 background environments so the noise floor is a test variable.
Do the benchmarks include per-vertical results?
No. Cekura's benchmarks measure turn-taking, interruption handling, latency, and consistency for one agent deployed unchanged across six platforms, at platform defaults rather than tuned configurations. That one agent is a healthcare appointment-scheduling assistant and there is no industry or vertical breakdown, so the figures indicate platform consistency and cannot be read as suitability for hospitality, retail, real estate, or restaurants.
Can one test suite cover several verticals?
Not as written. The scenario scaffolding and the platform integration carry across, but the assertions do not, because each vertical fails at a different layer. A hospitality suite asserting language persistence will pass a restaurant agent that attaches every modifier to the wrong line. Cekura reuses the scaffolding and swaps the assertions per vertical.
Test your agent against your vertical, not a generic script
Write your vertical's dominant failure as an assertion, expand it across the languages, accents, and background environments your callers actually bring, and gate on consistency across repeated runs. Book a Cekura demo to see a vertical suite built against your own agent.