Voice Agents for Sales and Support That Survive Production
How to deploy AI voice agents for sales and customer support with latency budgets, barge-in, telephony integration, approval gates, and CRM writes that hold up.

A voice agent can run production sales and support calls today, provided round-trip latency stays under roughly 800 ms end-to-end and every CRM write is idempotent, consent-gated, and logged. Breach either constraint and the artifact stops being an agent and starts being a liability.
What follows is the small set of design decisions that separate voice agents that survive peak-hour call volume from the demos that collapse the first time a customer interrupts. The constraints are not conceptual. They are measured in milliseconds, in SIP packets, in TCPA-compliant disclosures, and in whether the CRM write survives a dropped call.
The Latency Budget That Earns the Call
Human conversational turn-taking expects a response pause of roughly 200 ms. Anything past 800 ms registers as a lag, past 1.5 s as a line problem, past 3 s as a hang-up. That total budget has to cover audio capture, speech-to-text, LLM reasoning, any tool calls, text-to-speech, and the network round-trip back to the caller's phone. None of those stages are free.
A workable split looks like 150 ms for streaming ASR partials, 400 ms for first-token LLM latency, 150 ms for TTS first audio chunk, and 100 ms of network and jitter buffer. That leaves no room for a synchronous database lookup mid-sentence. The usual fix is to stream the LLM response token-by-token into the TTS, prefetch likely CRM records on turn start, and execute slow tool calls behind a spoken acknowledgement ("one moment while I pull that up") rather than silence. The physics is the same whether the caller is a retail prospect or an enterprise account, and the cheaper hops matter more than the glamorous ones; our note on race-to-zero latency covers the trade-offs in detail.
Barge-In and Turn-Taking Are Not Features
Barge-in — the agent stopping mid-sentence when the caller starts talking — is the single most reliable signal to a human that they are not on hold with an IVR. It requires full-duplex audio, server-side voice activity detection running on the inbound stream, and the ability to cancel in-flight TTS synthesis without leaving a half-played word on the line. Vendors that quote impressive latency numbers on a push-to-talk demo often fall apart here because their TTS cannot be interrupted deterministically.
The adjacent problem is endpointing: deciding when the caller has actually finished speaking. Fixed silence thresholds (say, 700 ms) produce either rude interruptions or awkward pauses. The production pattern is a model-based endpoint detector that uses prosody and partial-transcript semantics, tuned per language and per channel (mobile, landline, VoIP). Treat endpointing accuracy as a first-class metric alongside WER and first-token latency.
Telephony Integration via SIP and Platforms Like phony.ai
A voice agent that cannot answer a real phone number is a browser toy. Production telephony means SIP trunks, PSTN termination, DID provisioning, E.911 obligations in the US, STIR/SHAKEN attestation to avoid being flagged as spam, and codec negotiation (G.711 µ-law for most US PSTN, Opus where available). Platforms like phony.ai handle the carrier layer and expose a media stream the agent subscribes to, which removes months of work compared with building against a bare Asterisk or FreeSWITCH stack.
The integration question that matters is where the media stream terminates. Pulling raw PCM frames out of SIP into your own VAD and ASR pipeline gives you the latency control required above. Letting the telephony platform do ASR and hand you text saves engineering time but usually costs 100 to 300 ms and makes barge-in harder to tune. For regulated workloads the media path also determines where recordings are stored, which is a compliance decision, not an infrastructure one. The same logic that governs API hops in our Action Execution service applies to audio: every intermediary is latency, cost, and a point of custody.

CRM Writes Need Idempotency, Not Optimism
The hardest failure mode in a sales voice agent is not a wrong word, it is a right word followed by a wrong wire. A caller agrees to a demo slot, the agent says "booked," the line drops before the Salesforce write completes, the retry fires and double-books. Every side effect the agent can trigger — calendar holds, opportunity creation, case updates, outbound SMS, payment authorizations — needs an idempotency key tied to the call SID and the turn index, a transactional outbox that survives process restart, and a reconciliation job that reads the call recording after hangup to confirm what was actually promised.
The implication for design: the agent's prompt should not call the CRM directly. It should emit a structured intent (book_demo(prospect_id, slot_id)) that goes to a worker with retry, dead-letter, and manual override. For higher-risk actions — refunds, cancellations, anything billable — the worker should route to a human. Our guidance on approval gates and the companion piece on which agent actions actually need human approval cover where to draw that line.
Consent, Disclosure, and PCI Live in the Audio Path
The compliance surface for a phone-based agent is larger than for a chatbot because speech is a biometric and a phone call is a regulated transport. On February 8, 2024 the FCC confirmed that AI-generated voices fall under the TCPA, meaning outbound calls using synthetic voice need prior express consent, caller identification, and a working opt-out. In the EU, Article 50 of the AI Act requires any user interacting with an AI voice agent to be told so at the start of the interaction.
Consumer tolerance is narrower than executive expectations. An Upfirst analysis of 677,562 calls found that 80% of US consumers want AI receptionists to disclose their identity up front, and only 16% are comfortable with an AI processing payments. PCI DSS makes that limit concrete: if the agent takes a card number, the ASR transcript, the LLM context, and the call recording all become part of the cardholder data environment unless the audio is routed through DTMF suppression or a PCI-compliant pause-and-resume vendor during card entry. Most teams solve this by handing the caller off to a tokenized IVR for the payment leg, then resuming the agent. The recording consent regime (one-party vs two-party states, GDPR for EU callers) adds another branch to the call-flow logic that has to be versioned with the agent itself.
Where Voice Actually Pays Off
The economic case is not universal. McKinsey's work on generative AI in a 5,000-agent contact center reports a 14% lift in issue resolution per hour and 30 to 50% of sales-specialist time redirected from routine tasks to selling, but the same research notes those gains concentrate in high-volume, low-variance intents. Gartner's 2022 projection that conversational AI would reduce contact-center labor costs by $80 billion by 2026 was ambitious; the actual 2026 picture is more sober, with a Gartner survey finding only 4% of service leaders achieved the ROI they expected from AI.
The implication for sequencing: the voice workloads that pay back inside twelve months are outbound qualification (short scripts, clear disposition), appointment reminders, account lookups, and tier-one FAQ deflection. The ones that reliably fail a first-pass ROI test are complex troubleshooting, retention saves, and anything requiring joint reasoning across three or more backend systems. Scope the first voice agent accordingly.
The Real Trade-Off
Running voice at production quality means accepting a stricter engineering budget than text agents require: sub-second latency, deterministic interruption handling, telephony and recording compliance, idempotent side effects, and PCI-safe audio routing for any workflow that touches payment. That is a non-trivial amount of plumbing for a system whose first version will only handle the narrow intents where it earns its keep. The payoff is a channel that answers every call at 3 a.m., discloses what it is, hangs up cleanly, and leaves an audit trail a compliance officer will sign. For regulated sales and support teams under the executive pressure that 77% of support leaders now report, that trade is usually worth making — provided the first workload is one the physics and the regulators actually allow.
Timothy Carter is the Chief Revenue Officer. Tim leads all revenue-generation activities for marketing and software development activities. He has helped to scale sales teams with the right mix of hustle and finesse. Based in Seattle, Washington, Tim enjoys spending time in Hawaii with family and playing disc golf.
Put an agent to work, the right way.
Start on Automatic and put the workflow you want to automate in front of engineers who have shipped agents in regulated environments.
Agentic AI, in your inbox.
Occasional, high-signal notes on building and operating AI agents — automation patterns, architecture, and governance. No spam.


