Table of Contents
The voice sounds warm when it picks up, then the warmth drains out by minute three. The customer cuts in and the agent doesn't know where it left off, the account number comes out wrong, and the tone that felt right in the demo feels scripted on a real call.
Every team building a voice agent has run into this. The TTS layer works, right up until it meets a real conversation.
Today we're launching Flux TTS, the second model in the Flux family. Flux STT reimagined speech-to-text for live conversations. Flux TTS does the same for text-to-speech: the first conversation-native TTS built for real-time voice agents.
And, it's free until September 12.
[Start building →] [Try the demo →]
Why voice agents break the Text-to-Speech layer
Today's TTS was built for narration: audiobooks, voiceover, IVR prompts. But voice agents need a model that can keep up with a live conversation as it unfolds.
When you take a model built for the first job and drop it into the second, three things fall apart.
Tone resets every turn because the model doesn't know it's in a conversation. Every line is a fresh request, so an empathetic tone can't hold across a long call, and a shift in the customer's mood can't carry into the next response.
You end up doing the model's job for it. To get the delivery right, developers wrap everything in SSML, style tags, and prompt engineering, hand-feeding the model the context it should be picking up from the conversation itself.
And once a line starts generating, you can't change it, there is no slowing it down mid-sentence, no fixing pronunciation, no adjusting when the caller's mood shifts. Whatever the model started, it finishes, whether or not it still fits.
This is why we built Flux TTS to hold the whole conversation in memory as it speaks, instead of starting fresh on every line. Once the model can hear what's happening around it, a lot of things get easier:
- Automatic expressive delivery. No SSML, style tags, or prompt engineering.
- Cross-turn consistency. Tone, pacing, and pronunciation hold across the whole call.
- Native interruption handling. The server reports what the caller actually heard.
- Time to first audio as low as 80ms.
- Production-grade entity accuracy on the terms that break voice agents.
- Deploy anywhere: cloud, self-hosted, or on-prem.
Voices that read the room
What makes Flux TTS voices expressive by default, even without style tags?
A big piece of it comes down to training data. Instead of studio audio with style markup layered on top, Flux TTS learned from real human conversation, with all the disfluencies and the shifts in tone that happen when people actually talk to each other. So the voice comes out empathetic when a customer is frustrated and precise when the details matter, without anyone having to write a style tag to make it happen. The model picks up what's going on in the conversation and adjusts tone, pacing, and emotional register on its own.
The other key part is that the model keeps track of the whole conversation while it's speaking. That means the voice doesn't reset every turn like other TTS does. With most models, turn one comes out warm and slow, turn three sounds bright and clipped, and by turn five you're getting a flat scripted read. Flux TTS keeps one voice going the whole session, so the tone, pacing, and pronunciation you got at the start are still there at minute ten.
Accurate when precision matters
For businesses in healthcare, finance, and insurance, a voice agent lives or dies on the details: the drug name, the account number, the policy ID, the dollar amount. Those are the moments the customer is actually listening for, and the moments where getting it wrong costs the most.
The inputs that cause the most trouble in production are the specific ones: alphanumerics, drug names, technical strings, IVR patterns, currency, dates. These rarely surface during vendor demos, but they come up constantly in real calls. Flux TTS is built to handle them, with reliable delivery on the account numbers, order confirmations, and dosages voice agents actually have to say out loud.
The benchmarks back that up. Flux TTS has the lowest word error rate of any read-aloud TTS we tested, with a median WER of 2.2%, roughly half of ElevenLabs and a third of Cartesia. On hard prompts, the kind of inputs that come up in production, Flux TTS comes in at 3.4% and beats the next-best model by 47%.
Built for the shape of a conversation
Most TTS is one-way: text goes in, audio comes out. That works fine for narration, but it falls apart the moment a caller cuts in, a turn ends unexpectedly, or the agent needs to change speed or pronunciation while it's still speaking.
Flux TTS is built to participate in the conversation. Every utterance has an explicit lifecycle (Idle, Generating, Completing) with events the agent can act on. When a caller cuts in, the server reports exactly what the caller heard, so the agent resumes cleanly instead of guessing or repeating itself. Speed, pronunciation, and pacing can change while the model is speaking and server-side flushing lets the model decide when a phrase is ready, so developers don't have to write chunking logic to balance latency and naturalness themselves.
Three architectural choices make that work at real-time speed without sacrificing quality.
A high-fidelity neural codec. Most TTS uses a lossy codec that restricts the model's expressive range to avoid audio artifacts. Flux TTS uses a high-fidelity neural codec, the same family as recent research from Kyutai, Meta, and Google, so the full expressive range comes through without a fidelity cap.
Interleaved text-to-audio generation. Standard TTS processes the entire input before starting to generate audio, so first-audio latency grows with the length of the response. Flux TTS generates text and audio together, so audio starts streaming on the first token and first-audio latency stays under 200ms regardless of response length.
That matters because in a voice agent, the TTS shares a budget with the STT and the LLM, and a caller starts noticing lag around 800ms of total round-trip time.
A Mamba state-space backbone. Other models run on transformers, where holding conversational context gets expensive fast, so the model resets every turn. Flux TTS uses a Mamba state-space backbone with fixed-size memory, so it can hold context across the whole session without the quadratic cost.
Every other TTS on the market forces a tradeoff between expressiveness, stability, and latency: pick two, live without the third. These three choices are why Flux TTS doesn't. All the orchestration developers wire up themselves today, VAD, barge-in logic, state management, re-feeding conversation history to fake context, now lives in the model.
And it runs wherever your data has to live. Flux TTS deploys in the cloud, self-hosted, or on-prem, with the same model and same benchmarks across all three. HIPAA, data residency, and enterprise compliance are built in, so a voice agent in healthcare, finance, or government can run under the same rules the rest of the business does.
The full conversation loop: Flux STT + Flux TTS
You can wire up the best STT and the best TTS on the market, and the agent still ends up feeling stitched together. The two models don't share context, and the caller hears it as robotic: the tone doesn't shift when they get frustrated, slowing down has no impact on pace, and there are weird lags while they wait for a response.
Flux STT and Flux TTS are both built by Deepgram, purpose-built to work together. Today, they run as a paired stack behind one API, so wiring both into an agent is a single config. Next on the roadmap, the two models will share state directly. That's when the agent starts responding to how the caller actually sounds: matching their pace, shifting tone with their mood, pronouncing names back the way they were spoken.
That's the entire configuration. Flux STT on listen, Flux TTS on speak, your LLM of choice in the middle, and the agent runs.Together, Flux STT and Flux TTS make voice agents work in the environments that break most stacks.
Proven in restaurants
Take a restaurant. Between engine noise at a drive-thru, the person ordering at the kiosk next to you, a fryer hiss, and multilingual environments, it might be the hardest environment there is for a voice agent.
Restaurants also come with their own unique vocabulary and rhythm:
- Brand names: Dr. Pepper, Frappuccino
- Prices: $4.99, 2 for $10
- Non-English food names: açaí, queso, jalapeño
- Hours and locations: 8:00am to 8:00pm, San Diego, CA
- Confirmation pacing: repeating the order back at the speed the customer is thinking, not a scripted read
This is exactly the type of situation Flux TTS was designed to handle. In restaurant scenarios, it beats every model we tested – not just overall, but also on ability to maintain a consistent tone within the voice agent experience.
View the demo here.
What's next
Flux TTS ships at GA with expressive, conversational English voices. Coming next:
- More languages: dozens of them, with voice identity preserved across them
- Voice cloning: brand voices, customer voices, personalized agents, no engineering required
- Emotional controls: speaking style, non-verbal cues, dynamic range, without retraining
- Flux Shared State: Flux STT and Flux TTS sharing acoustic and semantic context, so what one model perceives conditions on what the other just produced
Our research team will publish technical deep dives on the pieces that make Flux TTS work, including the state-space architecture, interleaved generation, and cross-turn context modeling, in a companion series.
Try Flux TTS today
Flux TTS is available now in the Deepgram API, and free through September 12.
Deepgram's stance on the voice economy is simple: we meet builders where they already are. Flux TTS is available through REST and WebSocket APIs directly, and through the platforms most voice agent teams already use, including Pipecat, LiveKit, jambonz, and Twilio.
Get started:
Whether you're integrating Flux TTS into an existing voice agent stack or building fresh, the path is the same. Drop it in, wire up the conversational events, and ship.
