Build real-time voice applications with Gemini 3.8 Live and 3.5 Transcribe

Open original

Google launched two new speech models for developers: Gemini 3.8 Live handles real-time voice conversations with reasoning and background tool execution, while Gemini 3.5 Transcribe converts speech to text across 85+ languages with a 4.0% error rate.

  • Gemini 3.8 Live can execute API calls in the background while streaming audio responses, handle visual context, and support 97+ languages with accent consistency
  • Gemini 3.5 Transcribe achieves 4.0% word error rate (streaming) and 2.6% (non-streaming), with automatic code-switching and custom vocabulary biasing for domain-specific terms
  • Extended Thinking variant adds multi-step reasoning capabilities, ranking #1 on Artificial Analysis' Speech-to-Speech leaderboard
  • Google's full audio suite includes speech translation (70+ languages), text-to-speech, and music generation all available in the Gemini API

Google just released two new voice models for developers building real-time conversational AI. Gemini 3.8 Live handles speech-to-speech conversations while executing tasks in the background—you can call APIs and stream responses simultaneously without interrupting the user. The Extended Thinking version adds reasoning capabilities for complex problems and currently ranks first on Artificial Analysis' speech-to-speech leaderboard. Both models support 97+ languages, can parse alphanumeric data like confirmation codes accurately, and can process live visual inputs to ground conversations in what users are actually seeing. The other piece is Gemini 3.5 Transcribe, a dedicated speech-to-text model that hits 4.0% word error rate on streaming audio and 2.6% on non-streaming across 85+ languages. What makes it useful beyond raw accuracy: it handles code-switching (people mixing languages mid-sentence), lets you feed it custom vocabulary lists up to 1,000 terms for domain-specific jargon, and has a "smart mode" that cleans up transcripts by removing filler words and self-corrections. You can also use it for longer audio files up to an hour through their Interactions API, which includes speaker labeling and structured timestamps. Google's positioning this as a complete audio toolkit for developers. Beyond these two models, they've got speech-to-speech translation (70+ languages), a text-to-speech engine called Flash TTS, and Lyria 3.5 for music generation. Developers can jump in through ai.studio/live, grab example code from GitHub, or integrate via the Gemini API directly. The focus here is practical—these aren't research demos but production tools for building voice agents, call center systems, real-time captioning, and similar applications.

Ask about this article