The budget, written down
Reference / p50
Reference latency budget for a single voice agent turn
| Stage | Budget |
| SIP edge | 20 ms |
| Endpointing | 140 ms |
| Final ASR | 60 ms |
| Agent first token | 110 ms |
| TTS first audio | 100 ms |
| Playout | 50 ms |
| Turn | 480 ms |
Tool calls run in parallel with speech. A blocking lookup pushes the turn to about 730 ms; we design around it rather than hiding it.
Where it breaksFour failure modes
- Barge-in
- If TTS cannot stop inside a frame, the caller talks over the agent. We stop playback in under 60 ms and discard the queued audio.
- Endpointing
- A fixed silence timer fails on slow speakers and code-switching. We use acoustic and semantic cues, then let the caller resume.
- Tool time
- A 900 ms lookup is a bad conversation. We stream a holding phrase, cap the call at 800 ms, and design a fallback.
- Handoff
- A cold transfer loses the reason for the call. We pass the transcript, the entities and the attempted actions to the human queue.
Engineering targets, not marketing benchmarks. Every deployment is measured against its own budget.