TTS vs STT: Building a Voice Assistant with One API

A voice assistant is really two pipelines glued together: speech-to-text (STT) turns the user's audio into text your app can understand, and text-to-speech (TTS) turns your reply back into natural audio. The hard part is rarely either model in isolation, it is keeping both directions consistent, observable, and cheap at scale.

STT accuracy depends heavily on the domain. Generic models transcribe well-formatted English, but product names, technical jargon, and accented speech need evaluation on your own data. Collect a small evaluation set of real user utterances before committing: a 2% accuracy gain on your domain beats a headline benchmark every time.

TTS quality is about voice, not just correctness. Listen for prosody on long-form narration, pronunciation of brand terms, and how the voice handles numbers, timestamps, and filler sounds. Streaming matters too, users notice the difference between a 300ms first syllable and a 2-second wait for the full response.

Latency budget is the real constraint. Round-tripping audio through STT, your logic, and TTS should stay under a second for snappy assistants. That means choosing endpoints that stream partial transcripts and synthesize incrementally, and caching common responses so the same greeting does not get re-synthesized a thousand times.

Observability is the forgotten feature. Log per-request latency, model used, and cost for both directions. When a user complains about a 'robotic' reply, you want to know exactly which TTS voice and version produced it, not guess.

We built llm-kita so teams wire both directions with a single API key: STT and TTS endpoints, consistent OpenAI-compatible SDKs, and one bill. Two directions of voice, zero vendor sprawl.