# What is Voice Latency?

Also called Response Latency, Time to First Audio.

Voice latency is the delay between a speaker finishing and the system starting to respond audibly. It accumulates across the whole path: audio transport, endpointing, speech recognition, response generation, speech synthesis, and playback. Because spoken conversation has short natural gaps, latency is one of the strongest determinants of whether a voice interaction feels workable.

The budget is spent in identifiable places. Endpointing waits to be sure the speaker has stopped. Recognition finalizes the transcript. The reasoning step produces a reply, and any tool or database lookup adds its own time. Synthesis produces the first audio, and network transport adds delay at both ends. Measuring the total without breaking it down turns optimization into guesswork.

Practical techniques attack the perception as much as the number. Streaming recognition starts processing before the caller stops, generation streams into synthesis so speech begins on the first clause, and a brief acknowledgment covers a slow lookup. Lookups can also start speculatively as soon as intent is clear. Speech to speech models remove the intermediate text stage, which cuts a meaningful portion of the chain.

Averages hide the problem. A system with a good median and a heavy tail feels broken, because callers remember the pauses, so tail percentiles are the right target. Filler phrases used to cover delay stop working when they repeat. Latency also interacts with interruption handling, since a slow system invites the caller to start speaking again and both end up talking at once.

Text channels tolerate delay that voice does not. Several seconds of thinking is invisible in email and acceptable in chat with a typing indicator, but on a phone call it reads as a dropped line. The same assistant may therefore need a different answering strategy per channel rather than one shared response budget.

## Key points

- Measured from the speaker stopping to the first audible response
- Accumulates across transport, recognition, generation, and synthesis
- Streaming each stage hides much of the total delay
- Tail percentiles matter more than the average
- Text channels tolerate delays that voice does not

## In practice

A support line answers with a voice agent. The median response comes back in under a second, which callers accept, but one call in twenty stalls for four seconds while an order lookup runs. Those callers repeat themselves or hang up. Starting the lookup as soon as the order number is spoken, and speaking a short acknowledgment while it runs, removes most of the abandoned calls.

## Related terms

- [Voice Agent](/en/glossary/voice-agent)
- [Barge In](/en/glossary/barge-in)
- [Speech to Text](/en/glossary/speech-to-text)
- [Text to Speech](/en/glossary/text-to-speech)
- [Turn](/en/glossary/turn)

[Back to the AI Glossary](/en/glossary)