Name: FireRedVAD
Author: FireRedTeam

简介

FireRedVAD 是一款由 FireRedTeam 开发的高性能语音活动检测（VAD）模型。简单来说，它的核心能力是精准区分音频中的“人声”与“静音/噪音”，是构建语音助手、实时转写或 AI 电话等场景的必备前置环节。相比于传统的基于能量阈值的算法，该模型在复杂背景噪音下具有更强的鲁棒性，能有效降低误触发，提升端到端语音系统的响应速度。对于开发者而言，它部署简单，可作为 WebRTC 或 Whisper 等语音识别流水线的高效预处理插件，显著降低后续 ASR 模型的计算压力。

核心亮点

精准过滤背景噪音，高效提取纯净人声片段
极低延迟响应，适配实时语音交互场景
兼容 Apache-2.0 协议，企业级部署无压力
完美适配 ASR 流水线，降低语音识别计算开销

使用方法

安装依赖

# 安装 Hugging Face transformers
pip install transformers torch

SDK 使用

# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("FireRedTeam/FireRedVAD")
tokenizer = AutoTokenizer.from_pretrained("FireRedTeam/FireRedVAD")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引

pip install -U huggingface_hub

命令行下载

下载完整模型库

huggingface-cli download FireRedTeam/FireRedVAD

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

huggingface-cli download FireRedTeam/FireRedVAD config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('FireRedTeam/FireRedVAD')

Git 下载

请确保 lfs 已经被正确安装

Git 下载

git lfs install
git clone https://huggingface.co/FireRedTeam/FireRedVAD

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/FireRedTeam/FireRedVAD

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

pip install -U transformers torch

模型加载和推理

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('FireRedTeam/FireRedVAD')
tokenizer = AutoTokenizer.from_pretrained('FireRedTeam/FireRedVAD')

模型下载

我们推荐使用命令行或者 ModelScope SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 ModelScope：

操作指引

pip install modelscope

命令行下载

下载完整模型库

modelscope download --model FireRedTeam/FireRedVAD

下载单个文件到指定本地文件夹（以下载 README.md 到当前路径下 dir 目录为例）

modelscope download --model FireRedTeam/FireRedVAD README.md --local_dir ./dir

更多更丰富的命令行下载选项，可参见具体文档

SDK 下载

# 模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('FireRedTeam/FireRedVAD')

Git 下载

请确保 lfs 已经被正确安装

Git 下载

git lfs install
git clone https://www.modelscope.cn/FireRedTeam/FireRedVAD.git

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS

GIT_LFS_SKIP_SMUDGE=1 git clone https://www.modelscope.cn/FireRedTeam/FireRedVAD.git

ModelScope 模型页直接下载模型文件；无需将模型文件放在本站服务器。

Notebook 快速开发

下载并安装 ModelScope library

pip install "modelscope[audio,cv,nlp,multi-modal,science]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html

模型加载和推理

from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

p = pipeline('text-generation', 'FireRedTeam/FireRedVAD')

完整文档

来源: HuggingFace

---
language:

en

zh

license: apache-2.0
pipeline_tag: voice-activity-detection
tags:

voice-activity-detection

speech-activity-detection

audio-event-detection

vad

aed

streaming

non-streaming

audio

automatic-speech-recognition

asr

---

<div align="center">
<h1>
FireRedVAD: A SOTA Industrial-Grade
<br>
Voice Activity Detection & Audio Event Detection
</h1>

</div>

[[Paper]](https://huggingface.co/papers/2603.10420)
[[Code]](https://github.com/FireRedTeam/FireRedVAD)
[[HuggingFace]](https://huggingface.co/FireRedTeam/FireRedVAD)
[[ModelScope]](https://www.modelscope.cn/models/xukaituo/FireRedVAD)

FireRedVAD is a state-of-the-art (SOTA) industrial-grade Voice Activity Detection (VAD) and Audio Event Detection (AED) solution. It was introduced as part of FireRedASR2S.

FireRedVAD supports non-streaming/streaming VAD and non-streaming AED. It supports speech/singing/music detection in 100+ languages. Non-streaming VAD achieves 97.57% F1 on FLEURS-VAD-102, outperforming Silero-VAD, TEN-VAD, FunASR-VAD and WebRTC-VAD.

🔥 News

[2026.03.12] 🔥 We release FireRedASR2S technical report. See arXiv.

[2026.03.03] We release FireRedVAD as a standalone repository, along with model weights and inference code.

[2026.02.12] We release FireRedASR2S (FireRedASR2-AED, FireRedVAD, FireRedLID, and FireRedPunc) with model weights and inference code.

Method

DFSMN-based non-streaming/streaming Voice Activity Detection and Audio Event Detection.

Evaluation

FireRedVAD

We evaluate FireRedVAD on FLEURS-VAD-102, a multilingual VAD benchmark covering 102 languages.

FireRedVAD achieves SOTA performance, outperforming Silero-VAD, TEN-VAD, FunASR-VAD, and WebRTC-VAD.

|Metric\Model|FireRedVAD|Silero-VAD|TEN-VAD|FunASR-VAD|WebRTC-VAD|
|:-------:|:-----:|:------:|:------:|:------:|:------:|
|AUC-ROC↑ |99.60|97.99|97.81|- |- |
|F1 score↑ |97.57|95.95|95.19|90.91|52.30|
|False Alarm Rate↓ |2.69 |9.41 |15.47|44.03|2.83 |
|Miss Rate↓|3.62 |3.95 |2.95 |0.42 |64.15|

<sup>*</sup>FLEURS-VAD-102: We randomly selected ~100 audio files per language from FLEURS test set, resulting in 9,443 audio files with manually annotated binary VAD labels (speech=1, silence=0). This VAD testset will be open sourced (coming soon).

Note: FunASR-VAD achieves low Miss Rate but at the cost of high False Alarm Rate (44.03%), indicating over-prediction of speech segments.

Quick Start

Setup

1. Create a clean Python environment:

bash

$ conda create --name fireredvad python=3.10
$ conda activate fireredvad
$ git clone https://github.com/FireRedTeam/FireRedVAD.git
$ cd FireRedVAD  # or fireredvad

2. Install dependencies and set up PATH and PYTHONPATH:

bash

$ pip install -r requirements.txt
$ export PATH=$PWD/fireredvad/bin/:$PATH
$ export PYTHONPATH=$PWD/:$PYTHONPATH

3. Download models:

bash

# Download via ModelScope (recommended for users in China)
pip install -U modelscope
modelscope download --model xukaituo/FireRedVAD --local_dir ./pretrained_models/FireRedVAD
Download via Hugging Face
pip install -U "huggingface_hub[cli]"
huggingface-cli download FireRedTeam/FireRedVAD --local-dir ./pretrained_models/FireRedVAD

4. Convert your audio to 16kHz 16-bit mono PCM format if needed:

bash

$ ffmpeg -i <input_audio_path> -ar 16000 -ac 1 -acodec pcm_s16le -f wav <output_wav_path>

Script Usage

bash

$ cd examples
$ bash inference_vad.sh
$ bash inference_stream_vad.sh
$ bash inference_aed.sh

Command-line Usage

Set up PATH and PYTHONPATH first: export PATH=$PWD/fireredvad/bin/:$PATH; export PYTHONPATH=$PWD/:$PYTHONPATH

bash

$ vad.py --help $ vad.py --use_gpu 0 --model_dir pretrained_models/FireRedVAD/VAD --smooth_window_size 5 --speech_threshold 0.4 \ --min_speech_frame 20 --max_speech_frame 3000 --min_silence_frame 10 --merge_silence_frame 0 \ --extend_speech_frame 0 --chunk_max_frame 30000 --write_textgrid 1 \ --wav_path assets/hello_zh.wav --output out/vad.txt --save_segment_dir out/vad $ stream_vad.py --help $ stream_vad.py --use_gpu 0 --model_dir pretrained_models/FireRedVAD/Stream-VAD --smooth_window_size 5 --speech_threshold 0.3 \ --pad_start_frame 5 --min_speech_frame 8 --max_speech_frame 2000 --min_silence_frame 20 \ --chunk_max_frame 30000 --write_textgrid 1 \ --wav_path assets/hello_en.wav --output out/vad.txt --save_segment_dir out/stream_vad

$ aed.py --help $ aed.py --use_gpu 0 --model_dir pretrained_models/FireRedVAD/AED --smooth_window_size 5 --speech_threshold 0.4 \ --singing_threshold 0.5 --music_threshold 0.5 --min_event_frame 20 --max_event_frame 3000 \ --min_silence_frame 10 --merge_silence_frame 0 --extend_speech_frame 0 --chunk_max_frame 30000 --write_textgrid 1 \ --wav_path assets/event.wav --output out/aed.txt --save_segment_dir out/aed

Python API Usage

Set up PYTHONPATH first: export PYTHONPATH=$PWD/:$PYTHONPATH

#### Non-streaming VAD

python

from fireredvad import FireRedVad, FireRedVadConfig
vad_config = FireRedVadConfig(
    use_gpu=False,
    smooth_window_size=5,
    speech_threshold=0.4,
    min_speech_frame=20,
    max_speech_frame=2000,
    min_silence_frame=20,
    merge_silence_frame=0,
    extend_speech_frame=0,
    chunk_max_frame=30000)
vad = FireRedVad.from_pretrained("pretrained_models/FireRedVAD/VAD", vad_config)
result, probs = vad.detect("assets/hello_zh.wav")
print(result)
{'dur': 2.32, 'timestamps': [(0.44, 1.82)], 'wav_path': 'assets/hello_zh.wav'}

#### Streaming VAD

python

from fireredvad import FireRedStreamVad, FireRedStreamVadConfig
vad_config=FireRedStreamVadConfig(
    use_gpu=False,
    smooth_window_size=5,
    speech_threshold=0.4,
    pad_start_frame=5,
    min_speech_frame=8,
    max_speech_frame=2000,
    min_silence_frame=20,
    chunk_max_frame=30000)
stream_vad = FireRedStreamVad.from_pretrained("pretrained_models/FireRedVAD/Stream-VAD", vad_config)
frame_results, result = stream_vad.detect_full("assets/hello_en.wav")
print(result)
{'dur': 2.24, 'timestamps': [(0.28, 1.83)], 'wav_path': 'assets/hello_en.wav'}

#### Non-streaming AED

python

from fireredvad import FireRedAed, FireRedAedConfig
aed_config=FireRedAedConfig(
    use_gpu=False,
    smooth_window_size=5,
    speech_threshold=0.4,
    singing_threshold=0.5,
    music_threshold=0.5,
    min_event_frame=20,
    max_event_frame=2000,
    min_silence_frame=20,
    merge_silence_frame=0,
    extend_speech_frame=0,
    chunk_max_frame=30000)
aed = FireRedAed.from_pretrained("pretrained_models/FireRedVAD/AED", aed_config)
result, probs = aed.detect("assets/event.wav")
print(result)
{'dur': 22.016, 'event2timestamps': {'speech': [(0.4, 3.56), (3.66, 9.08), (9.27, 9.77), (10.78, 21.76)], 'singing': [(1.79, 19.96), (19.97, 22.016)], 'music': [(0.09, 12.32), (12.33, 22.016)]}, 'event2ratio': {'speech': 0.848, 'singing': 0.905, 'music': 0.991}, 'wav_path': 'assets/event.wav'}

FAQ

Q: What audio format is supported?

16kHz 16-bit mono PCM wav. Use ffmpeg to convert other formats: ffmpeg -i <input_audio_path> -ar 16000 -ac 1 -acodec pcm_s16le -f wav <output_wav_path>

Citation

bibtex

@article{xu2026fireredasr2s,
  title={FireRedASR2S: A State-of-the-Art Industrial-Grade All-in-One Automatic Speech Recognition System},
  author={Xu, Kaituo and Jia, Yan and Huang, Kai and Chen, Junjie and Li, Wenpeng and Liu, Kun and Xie, Feng-Long and Tang, Xu and Hu, Yao},
  journal={arXiv preprint arXiv:2603.10420},
  year={2026}
}

FireRedVAD

简介

核心亮点

使用方法

Hugging Face 下载

命令行下载

SDK 下载

Git 下载

PyTorch / Transformers 使用

模型下载

命令行下载

SDK 下载

Git 下载

Notebook 快速开发

完整文档

🔥 News

Method

Evaluation

FireRedVAD

Quick Start

Setup

Download via Hugging Face

Script Usage

Command-line Usage

Python API Usage

`{'dur': 2.32, 'timestamps': [(0.44, 1.82)], 'wav_path': 'assets/hello_zh.wav'}`

`{'dur': 2.24, 'timestamps': [(0.28, 1.83)], 'wav_path': 'assets/hello_en.wav'}`

`{'dur': 22.016, 'event2timestamps': {'speech': [(0.4, 3.56), (3.66, 9.08), (9.27, 9.77), (10.78, 21.76)], 'singing': [(1.79, 19.96), (19.97, 22.016)], 'music': [(0.09, 12.32), (12.33, 22.016)]}, 'event2ratio': {'speech': 0.848, 'singing': 0.905, 'music': 0.991}, 'wav_path': 'assets/event.wav'}`

FAQ

Citation

扫码进交流群