VieNeu TTS v3 Turbo

---
license: apache-2.0
datasets:

• pnnbao-ump/VieNeu-TTS-10k-ENVI

• vi

• en

• voice-cloning

• code-switching

• podcast

• emotion-control

• 48khz

🦜 VieNeu-TTS v3 Turbo


Overview

<video controls src="https://cdn-uploads.huggingface.co/production/uploads/68b923a86c86c127a1975eda/paPqSDpwFGrKtIZrqaEg4.mp4" width="100%"></video>

VieNeu-TTS v3 Turbo is the next generation of Vietnamese TTS — 48 kHz high-fidelity speech, 20 built-in preset voices across three regions (North / Central / South), instant voice cloning, real-time streaming, inline emotion cues, and seamless bilingual (En–Vi) code-switching.

The reference implementation is the vieneu Python SDK (v3.3.0). Its minimal install is torch-free: on CPU everything runs on ONNX Runtime (PyTorch is never imported), and on a CUDA machine it auto-switches to the PyTorch engine with automatic batching — same API, no code change.

> [!IMPORTANT]
> What's new in SDK v3.3.0:
> - 20 preset voices covering North / Central / South, both genders and several reading characters.
> - Torch-free voice cloning on CPU — cloning, denoising and add_voice now work on the ONNX-only install (kaldi-native-fbank + soxr), no PyTorch needed.
> - int8 backbone by default on CPU — ~1.6× faster and ~4× smaller than fp32 with quality preserved; use Vieneu(precision="fp32") for max fidelity.
> - Sliding-window repetition penalty for more stable long generations.

🏗️ Architecture & Credits

The VieNeu-TTS v3 Turbo architecture is an original design by the author, Phạm Nguyễn Ngọc Bảo, and is trained from scratch on ~10,000 hours of English–Vietnamese speech — it is not a fine-tune, distillation, or adaptation of any existing TTS model.

• Model architecture & training: designed and trained from scratch by Phạm Nguyễn Ngọc Bảo — https://github.com/pnnbao97

• Audio codec: MOSS-Audio-Tokenizer-Nano (OpenMOSS-Team) — 48 kHz neural audio codec.

• Phonemizer: sea-g2p — fast Vietnamese/English grapheme-to-phoneme, also by the author.

Tác giả: Phạm Nguyễn Ngọc Bảo

---

🔥 Quick Start (Web UI)

git clone https://github.com/pnnbao97/VieNeu-TTS.git
cd VieNeu-TTS

• Option 1: CPU & macOS (minimal, torch-free) — recommended — runs v3 Turbo via ONNX

uv sync

> ⚡ Use uv sync, not pip install, for the fastest CPU inference — it reproduces the locked environment with the optimized ONNX Runtime build. On Apple Silicon this ONNX/CPU path is faster than the MPS/PyTorch build.

• Option 2: GPU (CUDA ≥ 12.8) — v3 Turbo on GPU (PyTorch), batched automatically

uv sync --group gpu

Start the Web UI:

uv run vieneu-web

The UI opens at http://127.0.0.1:7860 with a Default voice tab, a Voice Cloning tab, and a Conversation tab (batched multi-speaker podcasts).

---

📦 Using the Python SDK (vieneu)

CPU (default) — torch-free, runs v3 Turbo via ONNX Runtime. Most users want this:

pip install vieneu

GPU (CUDA) — only if you have an NVIDIA GPU; install a CUDA build of PyTorch yourself first:

pip install torch==2.8.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128
pip install "transformers==4.57.6"   # Qwen3 backbone + MOSS codec (pinned — most stable)
pip install vieneu

> ℹ️ When is GPU actually worth it? The GPU win comes from batching, so it only pays off on long text (many chunks generated together in one forward — long-form or bulk synthesis). For short text the torch-free CPU/ONNX path is usually *faster*. Use CPU for short, interactive calls; reach for GPU for long-form or high-throughput work.

Full features guide

from vieneu import Vieneu
from time import time
Default = v3 Turbo (48 kHz). CPU → ONNX (torch-free, int8); GPU → PyTorch (auto-detected).
tts = Vieneu()                    # int8 backbone (default, fastest on CPU)
tts = Vieneu(precision="fp32")  # max fidelity, slower on CPU
text = """[cười] Trời ơi, cái giọng nó tự nhiên mà nó mượt mà dã man, nghe không khác gì người thật luôn. Giờ thì tha hồ mà quẩy content với cả kho giọng nói đa dạng, đủ mọi sắc thái biểu cảm. Mọi người bật loa lên rồi cùng trải nghiệm thử với mình nhé!"""
1. Default voice (Adam) — 48 kHz, no reference needed
start = time()
audio = tts.infer(text)
tts.save(audio, "output.wav")
print(f"Time taken: {time() - start:.2f} seconds")
2. Built-in voices by name
for label, voice_id in tts.list_preset_voices():
    print(label, voice_id)
audio = tts.infer("Mình là Xuân Vĩnh nè!", voice="Xuân Vĩnh")
tts.save(audio, "output_xuan_vinh.wav")
3. Emotion / non-verbal cues — EXPERIMENTAL: [cười] [thở dài] [hắng giọng]
audio = tts.infer("Nghe hay quá đi [cười]. Để mình nói tiếp [hắng giọng].", voice="Phạm Tuyên")
4. Instant voice cloning from a 3–8s reference clip (works on the torch-free CPU install too)
audio = tts.infer("Đây là giọng được nhân bản tức thì.", ref_audio="my_voice.wav", denoise=True)

> [!TIP]
> A temperature around 0.8 gives the most stable result for v3 Turbo. Higher values add expressiveness but can be less stable.

🔊 Real-time streaming

v3 Turbo streams frame-by-frame — first audio in ~300 ms, RTF < 1 on CPU (~2–3× realtime on a laptop, ~7× on Apple Silicon). Streaming runs on the ONNX/CPU engine; the GPU/PyTorch engine is built for batch throughput, not streaming, so pin backend="onnx" for realtime:

vieneu = Vieneu(backend="onnx")   # force ONNX/CPU — the streaming path (int8)
for chunk in vieneu.infer_stream("Xin chào các bạn!", voice="Adam"):
    play(chunk)   # np.float32 @ 48 kHz, play/write as it arrives

A full FastAPI streaming demo ships in apps/web_stream.py.

⚡ Batched generation (GPU)

infer_batch() runs many texts in one batched forward — same API on every backend (on CPU it still works, just sequentially). The batch caps at max_batch_size (default 32); pass batch_size=1 to disable. A single long infer() also auto-batches its own chunks.

audios = vieneu.infer_batch(texts, voice="Adam")   # or infer_batch(..., batch_size=64)

🦜 Voice cloning & saved voices

# Clone from a 3–8s clip; the reference is auto-denoised and trimmed to ≤ 8s
audio = vieneu.infer("Chào bạn, đây là giọng của tôi.", ref_audio="voice.wav", denoise=True)
Enroll once, then reuse by name like a built-in voice
vieneu.add_voice("Giọng của tôi", "voice.wav")
audio = vieneu.infer("Câu này dùng giọng đã lưu.", voice="Giọng của tôi")
Just clean up a clip (no synthesis)
wav, sr = vieneu.denoise("noisy.wav", out_path="clean.wav")

> denoise, add_voice and cloning work on every backend, including the torch-free CPU/ONNX install.

⚠️ Reading style is deprecated

style is still accepted by infer, infer_stream, infer_batch and add_voice so existing code keeps running, but it is ignored on v3 Turbo: the reading style is already baked into the reference itself (the speaker embedding + reference codes of the preset voice or of your cloned clip). Pick the reading character through the voice instead.

---

🎭 Preset Voices (20)

Call any of them