LongCat AudioDiT 3.5B tts text to speech SOTA

Name: LongCat AudioDiT 3.5B tts text to speech SOTA
Author: orbitalhd

Providerorbitalhd

Categoryaudio-generation

Licensemit

Downloads19

Stars0

Overview

LongCat AudioDiT 3.5B is a diffusion-transformer based text-to-speech model designed for high-fidelity audio synthesis. Moving away from traditional autoregressive architectures, it leverages a 3.5 billion parameter DiT backbone to achieve superior prosody and naturalness while reducing the common 'robotic' artifacts found in smaller TTS engines. For developers, this means a more robust tool for creating immersive voiceovers, virtual assistants, and dynamic game dialogue. It integrates well into pipelines requiring scalable, high-quality speech generation and offers a competitive alternative to proprietary APIs by providing a performant, open-weights solution under the MIT license.

Highlights

Diffusion Transformer architecture for high-fidelity speech synthesis
3.5B parameters ensuring nuanced prosody and natural intonation
MIT license allowing flexible commercial and private integration
State-of-the-art performance in text-to-audio generation
Reduced artifacts compared to standard autoregressive TTS models

Usage

Install

# Install Hugging Face transformers
pip install transformers torch

SDK Usage

# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("orbitalhd/LongCat-AudioDiT-3.5B-tts-text-to-speech-SOTA")
tokenizer = AutoTokenizer.from_pretrained("orbitalhd/LongCat-AudioDiT-3.5B-tts-text-to-speech-SOTA")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance

pip install -U huggingface_hub

CLI Download

Download the full repository

huggingface-cli download orbitalhd/LongCat-AudioDiT-3.5B-tts-text-to-speech-SOTA

Download a single file to a local folder (e.g. config.json into ./dir)

huggingface-cli download orbitalhd/LongCat-AudioDiT-3.5B-tts-text-to-speech-SOTA config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('orbitalhd/LongCat-AudioDiT-3.5B-tts-text-to-speech-SOTA')

Git Download

Make sure git-lfs is installed first

Git Download

git lfs install
git clone https://huggingface.co/orbitalhd/LongCat-AudioDiT-3.5B-tts-text-to-speech-SOTA

To skip LFS large-file downloads, use:

Skip LFS

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/orbitalhd/LongCat-AudioDiT-3.5B-tts-text-to-speech-SOTA

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

pip install -U transformers torch

Load the model and run inference

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('orbitalhd/LongCat-AudioDiT-3.5B-tts-text-to-speech-SOTA')
tokenizer = AutoTokenizer.from_pretrained('orbitalhd/LongCat-AudioDiT-3.5B-tts-text-to-speech-SOTA')

Full Documentation

来源: HuggingFace

---
license: mit
language:

tags:

text-to-speech

voice-cloning

voice-conversion

zero-shot-tts

SOTA

diffusion-transformer

speech-synthesis

audio-generation

neural-tts

commercial-use

pipeline_tag: text-to-speech
inference: false
---

⚡ LongCat-AudioDiT-3.5B — The Commercial SOTA for Voice Cloning (May 2026)

🎤 Overview

LongCat-AudioDiT-3.5B is the leading state-of-the-art (SOTA) model for zero-shot voice cloning and text-to-speech as of May 2026. Developed by Meituan's LongCat team, it delivers the most accurate voice cloning in the open-source domain, capturing not just tonal similarity but the unique way of speaking, breathing patterns, and subtle emotional inflections of the reference speaker.

✅ Voice Clone Quality: Exceptionally accurate — clones voice and speaking style with unmatched fidelity
✅ Commercial Use: Fully permitted under the MIT License (both code and model weights)
✅ Two Versions: 1B and 3.5B parameters (this card is for the 3.5B flagship version)

🏆 Model Ranking (May 2026)

Tier 1 — The Commercial SOTA

1. 🥇 LongCat-AudioDiT-3.5B (Meituan) — MIT License
- The undisputed best open-source voice cloning model. Unmatched in cloning accuracy, naturalness, and speaking style preservation.
2. 🥈 MOSS-TTS 8B (OpenMOSS / MOSI.AI) — Apache 2.0 License
- The only model that comes close. A production-grade 8B flagship with zero-shot voice cloning, long-form synthesis (up to ~1 hour), and fine-grained pronunciation control. Fully open-source and safe for commercial deployment.

⚠️ Technically Excellent but Excluded from Ranking

Fish Audio S2 Pro would compete for the #1 spot on technical merit alone. However, it is released under the Fish Audio Research License, which explicitly restricts usage to research and non-commercial purposes only. Any business or production use requires a separate paid license, making it unsuitable for unrestricted commercial projects. For this reason, it is excluded from this ranking.

📉 Everything Else

All other open-source TTS models — including but not limited to Qwen3-TTS, Voxtral TTS, VoxCPM2, OmniVoice, X-Voice, Chatterbox, and XTTS-v2 — do not match the voice cloning fidelity and naturalness of LongCat-AudioDiT-3.5B or MOSS-TTS 8B as of May 2026.

🧠 Architecture

LongCat-AudioDiT operates directly in a waveform latent space rather than the traditional mel-spectrogram domain. This single-stage Diffusion Transformer (DiT) approach eliminates the information loss inherent in spectrogram-based pipelines, preserving every subtle nuance of the reference speaker's voice.

LongCat-AudioDiT: High-Fidelity Diffusion Text-to-Speech in the Waveform Latent Space

Introduction

LongCat-AudioDiT is a state-of-the-art (SOTA) diffusion-based text-to-speech (TTS) model that directly operates on the waveform latent space.
> Abstract: We present LongCat-TTS, a novel, non-autoregressive diffusion-based text-to-speech (TTS) model that achieves state-of-the-art (SOTA) performance.
Unlike previous methods that rely on intermediate acoustic representations such as mel-spectrograms, the core innovation of LongCat-TTS lies in operating directly within the waveform latent space. This approach effectively mitigates compounding errors and drastically simplifies the TTS pipeline, requiring only a waveform variational autoencoder (Wav-VAE) and a diffusion backbone.
Furthermore, we introduce two critical improvements to the inference process: first, we identify and rectify a long-standing training-inference mismatch; second, we replace traditional classifier-free guidance with adaptive projection guidance to elevate generation quality.
Experimental results demonstrate that, despite the absence of complex multi-stage training pipelines or high-quality human-annotated datasets, LongCat-TTS achieves SOTA zero-shot voice cloning performance on the Seed benchmark while maintaining competitive intelligibility.
Specifically, our largest variant, LongCat-TTS-3.5B, outperforms the previous SOTA model (Seed-TTS), improving the speaker similarity (SIM) scores from 0.809 to 0.818 on Seed-ZH, and from 0.776 to 0.797 on Seed-Hard.
Finally, through comprehensive ablation studies and systematic analysis, we validate the effectiveness of our proposed modules.
Notably, we investigate the interplay between the Wav-VAE and the TTS backbone, revealing the counterintuitive finding that superior reconstruction fidelity in the Wav-VAE does not necessarily lead to better overall TTS performance.
Code and model weights are released to foster further research within the speech community.

This repository provides the HuggingFace-compatible implementation, including model definition, weight conversion, and inference scripts.

Experimental Results on Seed Benchmark

LongCat-AudioDiT obtains state-of-the-art (SOTA) voice cloning performance on the Seed-benchmark, surpassing both close-source and open-source modles.

| Model | ZH CER (%) ↓ | ZH SIM ↑ | EN WER (%) ↓ | EN SIM ↑ | ZH-Hard CER (%) ↓ | ZH-Hard SIM ↑ |
|:---|:---:|:---:|:---:|:---:|:---:|:---:|
| GT | 1.26 | 0.755 | 2.14 | 0.734 | - | - |
| Seed-DiT | 1.18 | 0.809 | 1.73 | **0

View official source