Qwen3 TTS Tokenizer 12Hz

提供商Qwen
分类audio-to-audio
许可证apache-2.0
下载量22.3K
星标7

简介

Qwen3 TTS Tokenizer 12Hz 是阿里 Qwen 团队推出的音频离散化分词工具，旨在将连续的音频信号转换为模型可理解的 Token 序列。它采用了 12Hz 的采样率，在保证语音关键特征（如语调、情感）不丢失的同时，极大地压缩了音频数据的长度。对于开发者而言，它是构建端到端语音大模型（Audio-to-Audio）的核心组件，解决了传统 TTS 方案中音频表征冗余的问题，让模型能像处理文本一样高效地处理音频，显著降低了推理延迟并提升了生成质量。

核心亮点

  • 12Hz 高压缩率，有效降低音频 Token 长度
  • 保留丰富语音细节，支持自然的情感表达
  • 端到端语音模型核心组件，加速推理响应
  • Apache-2.0 协议，对开发者极其友好

使用方法

安装依赖
# 安装 Hugging Face transformers
pip install transformers torch
SDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("Qwen/Qwen3-TTS-Tokenizer-12Hz")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-TTS-Tokenizer-12Hz")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引 
pip install -U huggingface_hub

命令行下载

下载完整模型库

下载完整模型库 
huggingface-cli download Qwen/Qwen3-TTS-Tokenizer-12Hz

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例） 
huggingface-cli download Qwen/Qwen3-TTS-Tokenizer-12Hz config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

SDK 下载 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('Qwen/Qwen3-TTS-Tokenizer-12Hz')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://huggingface.co/Qwen/Qwen3-TTS-Tokenizer-12Hz

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/Qwen/Qwen3-TTS-Tokenizer-12Hz

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

安装 Transformers 
pip install -U transformers torch

模型加载和推理

模型加载和推理 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('Qwen/Qwen3-TTS-Tokenizer-12Hz')
tokenizer = AutoTokenizer.from_pretrained('Qwen/Qwen3-TTS-Tokenizer-12Hz')

模型下载

我们推荐使用命令行或者 ModelScope SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 ModelScope：

操作指引 
pip install modelscope

命令行下载

下载完整模型库

下载完整模型库 
modelscope download --model Qwen/Qwen3-TTS-Tokenizer-12Hz

下载单个文件到指定本地文件夹（以下载 README.md 到当前路径下 dir 目录为例）

下载单个文件到指定本地文件夹（以下载 README.md 到当前路径下 dir 目录为例） 
modelscope download --model Qwen/Qwen3-TTS-Tokenizer-12Hz README.md --local_dir ./dir

更多更丰富的命令行下载选项，可参见具体文档

SDK 下载

SDK 下载 
# 模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('Qwen/Qwen3-TTS-Tokenizer-12Hz')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://www.modelscope.cn/Qwen/Qwen3-TTS-Tokenizer-12Hz.git

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://www.modelscope.cn/Qwen/Qwen3-TTS-Tokenizer-12Hz.git

ModelScope 模型页直接下载模型文件；无需将模型文件放在本站服务器。

Notebook 快速开发

下载并安装 ModelScope library

下载并安装 ModelScope library 
pip install "modelscope[audio,cv,nlp,multi-modal,science]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html

模型加载和推理

模型加载和推理 
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

p = pipeline('text-generation', 'Qwen/Qwen3-TTS-Tokenizer-12Hz')

完整文档

来源: HuggingFace

---
license: apache-2.0
pipeline_tag: audio-to-audio
tags:

  • audio

  • tts

  • speech

  • codec

---
---

Qwen3-TTS-Tokenizer-12Hz

This repository contains the Qwen3-TTS-Tokenizer-12Hz, as presented in the paper Qwen3-TTS Technical Report.

Qwen3-TTS-Tokenizer-12Hz achieves extreme bitrate reduction and ultra-low-latency streaming, enabling immediate first-packet emission through its 12.5 Hz, 16-layer multi-codebook design and a lightweight causal ConvNet.

Quickstart

Environment Setup

Install the qwen-tts Python package from PyPI:

bash
pip install -U qwen-tts

Tokenizer Encode and Decode

You can encode audio into discrete tokens for storage or transport and decode them back into speech using the snippet below:

python
import soundfile as sf
from qwen_tts import Qwen3TTSTokenizer

tokenizer = Qwen3TTSTokenizer.from_pretrained(
    "Qwen/Qwen3-TTS-Tokenizer-12Hz",
    device_map="cuda:0",
)


Encode audio from a URL (or local path)

enc = tokenizer.encode("https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-TTS-Repo/tokenizer_demo_1.wav")

Decode codes back into waveforms

wavs, sr = tokenizer.decode(enc)
sf.write("decode_output.wav", wavs[0], sr)

Overview

Introduction

<p align="center">
<img src="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-TTS-Repo/qwen3_tts_introduction.png" width="90%"/>
<p>

Qwen3-TTS covers 10 major languages (Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, and Italian) as well as multiple dialectal voice profiles. Key features:

  • Powerful Speech Representation: Powered by the self-developed Qwen3-TTS-Tokenizer-12Hz, it achieves efficient acoustic compression and high-dimensional semantic modeling of speech signals. It fully preserves paralinguistic information and acoustic environmental features.
  • Extreme Low-Latency Streaming Generation: Based on the innovative Dual-Track hybrid streaming generation architecture, it can output the first audio packet immediately after a single character is input, with end-to-end synthesis latency as low as 97ms.

Model Architecture

<p align="center">
<img src="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-TTS-Repo/overview.png" width="80%"/>
<p>

Released Tokenizers

| Tokenizer Name | Description |
|----------------|-------------|
| Qwen3-TTS-Tokenizer-12Hz | The Qwen3-TTS-Tokenizer-12Hz model which can encode the input speech into codes and decode them back into speech. |

Evaluation

For detailed evaluation results on speech generation consistency, speaker similarity, and tokenizer benchmarks (ASR tasks, PESQ, STOI, UTMOS), please refer to the technical report or the GitHub repository.

Citation

bibtex
@article{Qwen3-TTS,
  title={Qwen3-TTS Technical Report},
  author={Hangrui Hu and Xinfa Zhu and Ting He and Dake Guo and Bin Zhang and Xiong Wang and Zhifang Guo and Ziyue Jiang and Hongkun Hao and Zishan Guo and Xinyu Zhang and Pei Zhang and Baosong Yang and Jin Xu and Jingren Zhou and Junyang Lin},
  journal={arXiv preprint arXiv:2601.15621},
  year={2026}
}
查看官方来源