Gemini 3.

I've been messing around with the Gemini Live API for my macOS meeting translator, and I hit a massive wall regarding transcription. If you're looking into implementing real-time audio processing, don't just assume all "transcribe" models work the same way. Google dropped two models with names that are nearly identical, but they are fundamentally different tools for completely different workflows.

I spent way too much time digging through the docs to realize that the "real-time" part and the "who said what" part are currently mutually exclusive.

The Model Split: Live vs. Batch

The biggest headache is that the feature most people actually want—speaker diarization (identifying different speakers)—is completely absent from the streaming Live API. If you want to know if Speaker A or Speaker B is talking, you can't do it while the meeting is happening. You have to record the whole thing and send it as a batch via the standard Interactions API.

Here is the breakdown of how these two actually compare based on my testing:

• gemini-3.5-transcribe-live

- API Type: WebSocket streaming (Live API)
- Best use case: Real-time subtitles or live captions while someone is speaking.
- Speaker Diarization: Not supported.
- Timestamps: No word-level precision.
- Max Duration: 10 minutes per session.
- Pro: Provides interimInputTranscription so you see text appearing as the person speaks.

• gemini-3.5-transcribe

- API Type: Standard HTTP request (Interactions API)
- Best use case: Post-meeting minutes and formal logs.
- Speaker Diarization: Supports up to 8 different speakers.
- Timestamps: Provides detailed word-level timing.
- Max Duration: 1 hour (drops to 30 mins if using diarization).
- Pro: Much higher accuracy for structured documentation.

Implementing the Real-time Workflow

Since my app focuses on live translation, I had to stick with the gemini-3.5-transcribe-live model. To get this running, you have to change your configuration significantly compared to a standard translation setup. You need to set your responseModalities to TEXT and define your transcription preferences within the setup block.

Here is the specific JSON config I'm using for the real-time stream:

{
 "setup": {
 "model": "models/gemini-3.5-transcribe-live",
 "generationConfig": { "responseModalities": ["TEXT"] },
 "inputAudioTranscription": {
 "languageCodes": [],
 "mode": "SMART"
 },
 "realtimeInputConfig": {
 "automaticActivityDetection": { "disabled": false }
 }
 }
}

A quick tip on the languageCodes field: if you leave it as an empty array [], the model will attempt automatic language detection. If you know for a fact the meeting is in English, it's much safer to hardcode ["en-US"] to avoid detection lag.

I also highly recommend using "mode": "SMART" instead of "VERBATIM". Verbatim is a nightmare for meeting notes because it captures every single "uh," "um," and awkward pause. Smart mode acts like a light editor, cleaning up filler words and adding basic formatting so the text is actually readable.

One thing I'm keeping an eye on is the customVocabulary feature. It allows up to 1,000 entries (though the docs suggest staying under 100 for stability) to help the LLM recognize niche technical jargon or specific names. I haven't implemented it in my current build, but it'll be the first thing I add once the model starts tripping up on industry-specific acronyms.