Lightweight Speech Denoising.axera

提供商AXERA-TECH
分类speech-enhancement
许可证mit
下载量8
星标0

简介

Lightweight Speech Denoising.axera 是一款由 AXERA-TECH 推出的轻量化语音去噪模型。它专注于在保证语音清晰度的前提下，高效剔除背景环境噪音，非常适合对实时性要求较高且计算资源受限的端侧设备。对于开发者而言，该模型在部署难度上较低，能够无缝集成到语音助手、实时会议软件或智能穿戴设备的音频处理链路中，在提升语音识别（ASR）准确率前作为预处理环节具有极高实用价值。

核心亮点

  • 轻量化架构，极低延迟，适配端侧实时部署
  • 有效抑制环境噪音，提升语音纯净度与清晰度
  • MIT 协议开源，方便开发者快速集成与商用
  • 可作为 ASR 前端预处理，显著提升识别准确率

使用方法

安装依赖
# 安装 Hugging Face transformers
pip install transformers torch
SDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("AXERA-TECH/Lightweight-Speech-Denoising.axera")
tokenizer = AutoTokenizer.from_pretrained("AXERA-TECH/Lightweight-Speech-Denoising.axera")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引 
pip install -U huggingface_hub

命令行下载

下载完整模型库

下载完整模型库 
huggingface-cli download AXERA-TECH/Lightweight-Speech-Denoising.axera

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例） 
huggingface-cli download AXERA-TECH/Lightweight-Speech-Denoising.axera config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

SDK 下载 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('AXERA-TECH/Lightweight-Speech-Denoising.axera')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://huggingface.co/AXERA-TECH/Lightweight-Speech-Denoising.axera

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/AXERA-TECH/Lightweight-Speech-Denoising.axera

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

安装 Transformers 
pip install -U transformers torch

模型加载和推理

模型加载和推理 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('AXERA-TECH/Lightweight-Speech-Denoising.axera')
tokenizer = AutoTokenizer.from_pretrained('AXERA-TECH/Lightweight-Speech-Denoising.axera')

模型下载

我们推荐使用命令行或者 ModelScope SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 ModelScope：

操作指引 
pip install modelscope

命令行下载

下载完整模型库

下载完整模型库 
modelscope download --model AXERA-TECH/Lightweight-Speech-Denoising.axera

下载单个文件到指定本地文件夹（以下载 README.md 到当前路径下 dir 目录为例）

下载单个文件到指定本地文件夹（以下载 README.md 到当前路径下 dir 目录为例） 
modelscope download --model AXERA-TECH/Lightweight-Speech-Denoising.axera README.md --local_dir ./dir

更多更丰富的命令行下载选项，可参见具体文档

SDK 下载

SDK 下载 
# 模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('AXERA-TECH/Lightweight-Speech-Denoising.axera')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://www.modelscope.cn/AXERA-TECH/Lightweight-Speech-Denoising.axera.git

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://www.modelscope.cn/AXERA-TECH/Lightweight-Speech-Denoising.axera.git

ModelScope 模型页直接下载模型文件；无需将模型文件放在本站服务器。

Notebook 快速开发

下载并安装 ModelScope library

下载并安装 ModelScope library 
pip install "modelscope[audio,cv,nlp,multi-modal,science]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html

模型加载和推理

模型加载和推理 
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

p = pipeline('text-generation', 'AXERA-TECH/Lightweight-Speech-Denoising.axera')

完整文档

来源: HuggingFace

---
license: mit
language:

  • zh

  • en

pipeline_tag: audio-to-audio
tags:
  • speech-enhancement

---

Lightweight-Speech-Denoising

This project provides lightweight speech enhancement (denoising) models optimized for Axera NPU platforms, combining the DSP framework of RNNoise and the model architecture of GTCRN.

Key Features

  • Good Denoising Quality — Built on GTCRN and RNNoise frameworks; strong noise suppression even at very low parameter counts.
  • Ultra-Lightweight Models — Smallest model under 100 KB; CMM memory footprint below 150 KB.
  • Minimal Operators — tiny_v5 and conv_se are pure convolutional models with very few operator types; tiny_v5 supports quantization on the operator-limited AX525 platform.
  • Multi-Platform Support — Board inference validated on AX620Q, AX630C, and AX650; PTQ quantization also available for AX620L, AX637, and AX525.

Convert tools links

For those interested in model conversion, refer to:

Support Platform

  • AX650
- M4N-Dock（爱芯派Pro） - M.2 Accelerator Card
  • AX630C
- 爱芯派2 - Module-LLM - LLM630 Compute Kit
  • AX620Q
  • AX620L
  • AX637
  • AX525（only for tiny_v5）

How to use

Directory layout on device:

code
root@ax650:~# tree Lightweight-Speech-Denoising.axera
Lightweight-Speech-Denoising.axera
├── README.md
├── axmodels
│   ├── ax525_tiny_v5_setrain.axmodel
│   ├── ax620E_conv_se_setrain.axmodel
│   ├── ax620E_gtcrn_setrain.axmodel
│   ├── ax620E_tiny_v5_setrain.axmodel
│   ├── ax620L_conv_se_setrain.axmodel
│   ├── ax620L_gtcrn_setrain.axmodel
│   ├── ax620L_tiny_v5_setrain.axmodel
│   ├── ax637_conv_se_setrain.axmodel
│   ├── ax637_gtcrn_setrain.axmodel
│   ├── ax637_tiny_v5_setrain.axmodel
│   ├── ax630c_conv_se_setrain.axmodel
│   ├── ax630c_gtcrn_setrain.axmodel
│   ├── ax630c_tiny_v5_setrain.axmodel
│   ├── ax650_conv_se_setrain.axmodel
│   ├── ax650_gtcrn_setrain.axmodel
│   └── ax650_tiny_v5_setrain.axmodel
├── build_ax620q
│   └── test_se_denoise_ax
├── build_ax630c
│   └── test_se_denoise_ax
├── build_ax650
│   └── test_se_denoise_ax
├── models
│   ├── conv_se_ax650_config.ini
│   ├── gtcrn_7input_ax650_config.ini
│   ├── tiny_v5_ax650_config.ini
│   └── ...
├── run_ax620q_all.sh
├── run_ax630c_all.sh
├── run_ax650_all.sh
└── test_wavs
    └── mix.wav

Download all files from this repository to the device, then run the corresponding script for your platform:

bash
# AX650
sh run_ax650_all.sh

AX630C

sh run_ax630c_all.sh

AX620Q

sh run_ax620q_all.sh

Output .wav files will be saved to output/<platform>_all/.

Inference Results

tiny_v5

| Platform | Avg Infer (ms) | RTF | Realtime Speedup |
|----------|---------------|--------|-----------------|
| AX650 | 0.160 | 0.0117 | 85.3x |
| AX630C | 0.587 | 0.0232 | 43.1x |
| AX620Q | 0.736 | 0.0332 | 30.1x |
| AX620L | TBD | TBD | TBD |
| AX637 | TBD | TBD | TBD |
| AX525 | TBD | TBD | TBD |

conv_se

| Platform | Avg Infer (ms) | RTF | Realtime Speedup |
|----------|---------------|--------|-----------------|
| AX650 | 1.963 | 0.0365 | 27.4x |
| AX630C | 7.803 | 0.1092 | 9.2x |
| AX620Q | 14.938 | 0.1970 | 5.1x |
| AX620L | TBD | TBD | TBD |
| AX637 | TBD | TBD | TBD |

GTCRN

| Platform | Avg Infer (ms) | RTF | Realtime Speedup |
|----------|---------------|--------|-----------------|
| AX650 | 2.766 | 0.1756 | 5.7x |
| AX630C | 2.835 | 0.1820 | 5.5x |
| AX620Q | 3.535 | 0.2295 | 4.4x |
| AX620L | TBD | TBD | TBD |
| AX637 | TBD | TBD | TBD |

> Test audio: mix.wav, duration 9.77s, 16kHz mono.

TODO

  • [ ] AX525 board inference (quantization done)
  • [ ] AX620L board inference (quantization done)
  • [ ] AX637 board inference (quantization done)

References

  • RNNoise — Mozilla open-source DSP + RNN noise suppression framework; STFT/iSTFT and kiss_fft implementation reused in this project.
https://github.com/xiph/rnnoise
  • GTCRN — Lightweight Gated Temporal Convolutional Recurrent Network for speech enhancement.
https://github.com/Xiaobin-Rong/gtcrn
查看官方来源