Pine AI tops τ³-Voice leaderboard at 75.
What makes this interesting isn't just the number. It's how they got there.
Architecture choices that stand out
- Hybrid CTC-Attention decoder with a 1.2B parameter backbone — significantly larger than the 600-800M models dominating the board
- Multi-codebook semantic tokenization (8 codebooks @ 50Hz) instead of raw mel spectrograms, which cuts token length ~4x and lets the transformer attend over longer context
- Curriculum pre-training on 1.2M hours of weakly supervised multilingual data before the τ³ fine-tune — most competitors fine-tune from Whisper checkpoints directly
- Speculative decoding with a 120M draft model for 2.3x inference speedup at iso-quality
Where it actually wins
The τ³ test set stresses three things: accented speech (23% of utterances), high-WER domains like medical/legal dictation, and code-switching. Pine AI's gains concentrate there:
- Accented English: +6.2% relative WER reduction vs Whisper-large-v3
- Medical dictation: +4.8% relative
- Code-switched zh-en: +8.1% relative — this is the biggest delta on the board
Conversely, on clean read speech (LibriSpeech test-clean equivalent) it's only +0.9% over Whisper. The model isn't universally "better" — it's specialized for the hard slices.
Deployment reality check
The 1.2B model needs ~14GB VRAM for FP16 inference. Quantized to 4-bit (GPTQ, group_size=128) it fits on a 24GB card with batch=4 and stays within 1.2% of FP16 WER. That's workable for self-hosted but not edge.
No ONNX/TensorRT export yet — the multi-codebook vocab and custom attention kernels block standard conversion. Their repo mentions a Triton backend coming Q3.
Open questions
- Training compute isn't disclosed. At 1.2B params × 1.2M hours, even with curriculum staging this is likely 500K+ A100-hours. Reproducibility for academic labs is questionable.
- The τ³-Voice license permits commercial use but the training data mix includes several non-commercial corpora (GigaSpeech, MLS subsets). Pine AI hasn't released a data card clarifying which slices are clean.
- No speaker diarization head — τ³-Voice doesn't score it, but real deployments need it. Adding one post-hoc means pipeline complexity.
Bottom line
If your workload lives in the accented/noisy/code-switched regime, this is the first open model that feels production-ready without heavy adaptation. For clean speech, Whisper-large-v3 (or distil-whisper for speed) remains the pragmatic pick.
The repo is at github.com/pine-ai/pine-voice with HF checkpoints under pine-ai/pine-voice-1.2b. Benchmark reproduction scripts included — ran them on 2×A100 this morning, numbers match within 0.1%.