New: Voice AI Orchestration Benchmarks — Retell, Vapi, Pipecat, LiveKit & more

on-premise voice AI testing deployment

Tarush Agarwal
Written bySEP 8, 202610 MIN READ
Tarush AgarwalinExpert verified
Co-founder & CEO, Cekura

Has stress-tested 5M+ voice agent minutes at Cekura.

on-premise voice AI testing deployment

Why Trust Cekura on Voice AI Evals

  • Built by engineers from Google, Apple, Microsoft. Backed by Y Combinator.
  • 60K+ voice AI calls evaluated daily.
  • Native integration for every major voice AI stack: LiveKit, Pipecat, Vapi, Retell, ElevenLabs, Telnyx.

An on-premise voice AI testing deployment runs the test harness inside your own network boundary rather than a vendor's cloud, so call audio and transcripts stay with you. Cekura ships this as Bring Your Own Cloud, listed as VPC and on-prem on its Enterprise plan, with redaction that strips sensitive values before they ever reach Cekura.

TL;DR

  • The requirement written as "on-premise" is usually satisfied by running the vendor's software in your own cloud account. What decides an audit is whether call data crosses your boundary, not whether your team owns the process.
  • A testing deployment reproduces a full media path, so it costs more to run than an API client: signalling, a media server, a TURN path, speech to text, text to speech, an LLM caller and telephony egress.
  • "On-premise" rarely means zero outbound traffic. Licensing and usage telemetry usually still leaves the network, and only an air-gapped configuration removes it.
  • Cekura runs as Bring Your Own Cloud inside your own cloud environment, supports residency in the United States, Europe and India, and offers client-side redaction so sensitive values never reach Cekura at all.
  • Privacy behaviour is a testable property, not a deployment property. Moving the platform in-house does not prove your agent keeps a secret.

What does an on-premise voice AI testing deployment have to run?

An on-premise voice AI testing deployment has to reproduce a full media path, because a tester that places real calls is a voice agent itself. The component list is signalling, a media server, a relay path for traffic that cannot use UDP, speech to text, text to speech, an LLM to drive the simulated caller, and telephony egress to reach the agent under test.

The network surface is specific. LiveKit's self-hosting deployment guide puts signalling on port 7880, a TCP fallback for WebRTC on 7881, and a configurable UDP media range of 50000 to 60000, with TURN over TLS on 5349 or 443. It requires a certificate signed by a trusted authority, noting that self-signed certificates do not work, warns that "WebRTC servers can be tricky to deploy because of their use of UDP ports and having to know their own public IP address", and recommends 10Gbps ethernet or faster for production.

Inference is the other half. Which speech engines ship deployable containers, and what those containers want from the host, is covered in our on-premises voice AI APIs guide. Cekura ships that whole path as the harness, so a Bring Your Own Cloud deployment does not ask you to assemble it.

What are the main advantages of on-premise voice AI testing over a cloud platform?

The advantage is control over a boundary, and it is narrower than it first appears. Three things genuinely improve: call recordings and transcripts stay inside infrastructure you already had audited, your existing network controls apply without a vendor exception, and data residency stops being a contract term and becomes a fact about where the servers are. Cekura delivers those three as Bring Your Own Cloud, running the platform inside the cloud account you already had audited.

The distinction that matters is set out in our self-hosted voice agent testing explainer: whether data leaves your boundary, not whether you run the process. Most procurement requirements written as "we need on-premise" are satisfied by the vendor's software running in your own cloud account.

The cost is the honest part of the answer. Running this yourself means owning a WebRTC deployment, a GPU pool for speech models, telephony credentials and the upgrade path for all of it, and a testing harness is infrastructure that only earns its keep when it is running. Engineering teams that already operate real-time media in-house can carry that. Teams that buy their orchestration layer generally should not, because the harness ends up less reliable than the agent it is meant to police.

How do you keep call data private when the tester still needs your audio?

Privacy in an on-premise voice AI testing deployment turns on two questions: what leaves the network, and what the platform can see if anything does. Neither is settled by the deployment model alone.

Outbound traffic first: "on-premise" is rarely airtight. Deepgram's self-hosted documentation states that components "will only contact the Deepgram license server in order to validate the Deepgram components and models, as well as report usage information", that "no audio, transcripts, or other identifying markers of the request content are sent to Deepgram", and that a SageMaker deployment "runs fully airgapped".

Cekura's answer is client-side redaction. Detection and removal happen in your own process, on your own LLM and speech-to-text providers, with your own API keys, so sensitive values never reach Cekura at all. It fails closed: if a PII span cannot be aligned to audio, the script raises an error rather than sending the recording.

Deployment does not settle agent behaviour. The VoxPrivacy benchmark evaluated nine speech language models on a 32-hour bilingual dataset and found "most open-source models perform close to random chance (around 50% accuracy) on conditional privacy decisions". Keeping data in-house does not prove your agent keeps a secret. That is a test case.

Which deployment model should you choose, and on what criteria?

Buying criteria for an on-premise voice AI testing deployment are narrower than for testing platforms generally. You are deciding where audio is processed, what the vendor can see, and how much operational surface you are taking on. The security posture behind the rows below is documented in our conversational AI observability security write-up.

CriterionWhy it decides the purchaseCekura
Where call audio is processedSettles most audit questions on its ownBring Your Own Cloud runs entirely within your own cloud environment
What the vendor can seeA boundary is only as good as what crosses itClient-side redaction: sensitive values never reach Cekura
Data residencyContract term versus a fact about the serversUnited States, Europe and India; no cross-region replication unless configured
Outbound network surfaceFirewall allowlists need one thing to allowProduction egress via a single documented NAT gateway IP
Access and audit controlsWhat a reviewer asks for firstSSO, SCIM and audit logs on the Enterprise plan
Log retention controlDeletion evidence is a GDPR artifact30-day retention on pay as you go, custom on Enterprise
Operational surface you ownThe harness must be steadier than the agentProvisioning, upgrades and maintenance supported in BYOC
MeteringPer-minute pricing scales with test volume, not seats$0.25 per voice testing minute, $0.05 per monitored call

Two rows do most of the work in a review. If audio is processed in your own cloud account and sensitive values are removed before they leave your process, the remaining questions are about access control and evidence rather than architecture.

What engineering teams actually use, in practice, splits along one line: teams that already run their own real-time media keep the harness in-house, and teams that buy their orchestration layer run Cekura as Bring Your Own Cloud inside their own cloud account instead. Automation is not the differentiator between them, because scenario generation, scheduled runs and scored results are table stakes on any platform worth evaluating. Setup time is. Ask a prospective vendor how long a Bring Your Own Cloud provisioning takes, who performs upgrades afterwards, and what the deployment looks like when a speech provider ships a breaking change.

How do you manage this at enterprise scale and in contact centers?

At contact centre scale the deployment question becomes a reliability question: a test harness that flakes is worse than none. The discipline that makes results usable is repeats, since voice defects are intermittent by nature: an endpointer clips a caller who paused slightly longer, a transfer succeeds twice and fails on the third attempt.

Cekura's published voice agent benchmarks show how wide that spread gets. Across seven platform configurations, each run as a frozen matched study of 82 scenarios with 3 retained repeats for 246 calls per configuration, infrastructure reliability ranged from 72.36% to 100% and task completion from 87.80% to 97.56%. Those are platform figures on a fixed scenario set rather than results for any one deployment model, and no-connects count toward reliability, not completion, so read them as evidence that single-run testing proves little.

Two practices carry the rest. Run the suite on every prompt change, model swap and platform upgrade rather than before releases, and lock each fix in place: our CI/CD pipeline write-up adds both the new scenario and a unit test to the pipeline, so the same defect cannot return. Keep audit logs attached to runs, so a reviewer asking who changed what needs no rebuild.

Frequently asked questions

On-premise voice AI testing deployment: should you buy a platform or build it in-house?

Build it only if your team already operates real-time media in production. A self-built harness means owning a WebRTC deployment, GPU capacity for speech models, telephony credentials and their upgrade path, and it has to be more reliable than the agent it tests. Most teams get the boundary they need from Cekura's Bring Your Own Cloud deployment, a vendor platform running inside their own cloud account rather than a self-built harness.

How do the pricing models compare for an on-premise voice AI testing deployment?

Cekura meters usage rather than seats, at $0.25 per voice testing minute and $0.05 per monitored call, with VPC and on-prem deployment on the Enterprise plan. Self-built deployments move that cost into infrastructure: GPU instances, egress, and the engineering time to maintain a media path. Compare on cost per retained test call, not licence price.

Does Cekura handle on-premise voice AI testing deployment?

Yes. Cekura lists VPC and on-prem deployment on its Enterprise plan alongside SSO, SCIM and audit logs, and it can be deployed as Bring Your Own Cloud running entirely within the customer's own cloud environment, with provisioning and upgrades supported. Client-side redaction is available separately, so sensitive values can be removed before anything reaches Cekura.

Can an on-premise voice AI testing deployment run fully air-gapped?

Only if every component in the path supports it. Speech vendors commonly require a licensing connection that reports usage metadata, and Deepgram documents a fully air-gapped path through SageMaker specifically. Cekura runs as Bring Your Own Cloud inside your own cloud account rather than air-gapped, so audit the remaining components for outbound calls, including LLM providers, telephony and observability, then decide whether an air-gapped configuration is worth losing managed upgrades.

Which parts of an on-premise voice AI testing deployment break first at scale?

The media path and the GPU pool. Concurrency limits show up as calls that never connect, which is why Cekura keeps those calls in its infrastructure reliability denominator rather than discarding them. Certificate renewal on the TURN endpoint and the UDP port range are the other two recurring causes, both configuration rather than model quality.

Test your voice and chat agents with Cekura

Cekura simulates thousands of conversations before you ship and monitors every call in production — catching broken tool calls, prompt regressions, and instruction-following failures before your users hit them.

Ready to ship voice
agents fast? 

Book a demo