Appendix A: Provider Comparison Matrix

The tables below compare major providers across the three pillars of a voice AI pipeline: speech-to-text (STT), large language models (LLM), and text-to-speech (TTS). Prices and performance characteristics are current as of early 2026 and change frequently. Use this as a starting point for evaluation, then benchmark with your own traffic.

All latency figures are measured from a US-East server with warm connections. Cold start latencies are typically 100-300ms higher.

Speech-to-Text (STT)

Recommendation: Deepgram Nova-3 is the default choice for English voice agents. It has the lowest latency, strong accuracy on phone-quality audio, and the best support for interim (partial) results, which lets the LLM start processing before the speaker finishes. AssemblyAI is a solid alternative with better accuracy on accented speech. Gladia wins on language breadth if you need coverage beyond the top 20 languages.

Key considerations:

• Interim results: Essential for low-latency agents. Deepgram and AssemblyAI both support this well.
• Phone audio: 8kHz, mono, often compressed. Test accuracy on real phone recordings, not studio-quality samples.
• Endpointing: Deepgram provides server-side endpointing, but most production agents use LiveKit’s built-in VAD/EOU model instead.
• Profanity and redaction: If your use case requires PII filtering, check whether the provider offers server-side redaction.

Large Language Models (LLM)

Recommendation: gpt-4.1-nano is the workhorse for production voice agents. Its 150ms TTFT is fast enough for natural conversation, tool calling is reliable, and the cost is negligible at scale. Use gpt-4.1-mini when you need better reasoning (complex scheduling logic, nuanced disambiguation). Reserve gpt-4o or Claude for offline analysis tasks like call quality evaluation.

Key considerations:

• TTFT matters more than throughput for voice. A model that generates tokens at 200 tok/s but takes 500ms to start is worse than one at 100 tok/s with a 150ms start.
• Tool calling reliability: OpenAI models have the most robust tool calling. DeepSeek occasionally malforms tool call JSON on complex schemas. Cerebras models running open-source Llama have weaker tool calling.
• Streaming: All providers listed support streaming. Verify that your SDK handles streamed tool calls correctly, since some providers send tool call arguments incrementally.
• DeepSeek for CI: At $0.14/1M input tokens, DeepSeek is ideal for running integration tests in CI pipelines. It supports the OpenAI API format, so switching requires only a base URL and API key change.

Text-to-Speech (TTS)

Recommendation: Cartesia Sonic 2 is the default for latency-sensitive voice agents. Its sub-100ms first-byte latency is noticeably faster than alternatives, and the voice quality is excellent for phone conversations. ElevenLabs has slightly more natural prosody and better language coverage, but the higher latency is audible. Callers experience a longer pause before the agent starts speaking.

Key considerations:

• First-byte latency is the critical metric. Total generation time matters less because audio streams to the caller as it is produced. A provider that starts in 80ms and takes 2 seconds total sounds faster than one that starts in 300ms and takes 1.5 seconds total.
• Voice selection: Spend time choosing the right voice. A calm, clear, mid-pitch voice works best for business phone calls. Avoid voices that sound overly enthusiastic or robotic.
• Phone codec compatibility: Phone calls use narrow-band codecs (G.711, 8kHz). High-fidelity TTS features like whisper and emphasis may not survive codec compression. Test with actual phone calls, not browser playback.
• Character cost: Shorter agent responses save money. An agent that averages 40 characters per response costs half as much in TTS as one averaging 80 characters. Prompt engineering for conciseness is a cost optimization strategy.

Putting It Together

The combination we use in production and recommend as a starting point:

This stack delivers end-to-end latency under 600ms on a well-provisioned machine, at a cost of roughly $0.02-0.04 per minute of conversation.