distilbart mnli 12 1

提供商valhalla
分类zero-shot-classification
许可证Apache-2.0
下载量82.6K
星标0

简介

distilbart-mnli-12-1 是一款轻量级的零样本分类（Zero-Shot Classification）模型。它基于 BART 架构并经过蒸馏优化，核心能力在于无需针对特定标签进行训练，即可直接对文本进行类别判定。对于开发者而言，它解决了传统分类模型需要大量标注数据和重新训练的痛点，非常适合用于快速构建原型、处理长尾分类场景或在计算资源有限的边缘端部署。如果你需要一个比大型 LLM 更快、比传统分类器更灵活的文本分拣工具，它是极佳的选择。

核心亮点

  • 无需训练数据即可实现自定义类别文本分类
  • 经过模型蒸馏，推理速度快且部署成本低
  • 适用于自动化标签分发和内容快速预筛选
  • Apache-2.0 协议，支持商业化自由集成

使用方法

安装依赖
# 安装 Hugging Face transformers
pip install transformers torch
SDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("valhalla/distilbart-mnli-12-1")
tokenizer = AutoTokenizer.from_pretrained("valhalla/distilbart-mnli-12-1")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引 
pip install -U huggingface_hub

命令行下载

下载完整模型库

下载完整模型库 
huggingface-cli download valhalla/distilbart-mnli-12-1

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例） 
huggingface-cli download valhalla/distilbart-mnli-12-1 config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

SDK 下载 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('valhalla/distilbart-mnli-12-1')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://huggingface.co/valhalla/distilbart-mnli-12-1

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/valhalla/distilbart-mnli-12-1

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

安装 Transformers 
pip install -U transformers torch

模型加载和推理

模型加载和推理 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('valhalla/distilbart-mnli-12-1')
tokenizer = AutoTokenizer.from_pretrained('valhalla/distilbart-mnli-12-1')

完整文档

来源: HuggingFace

---
datasets:

  • mnli

tags:
  • distilbart

  • distilbart-mnli

pipeline_tag: zero-shot-classification
---

DistilBart-MNLI

distilbart-mnli is the distilled version of bart-large-mnli created using the No Teacher Distillation technique proposed for BART summarisation by Huggingface, here.

We just copy alternating layers from bart-large-mnli and finetune more on the same data.

| | matched acc | mismatched acc |
| ------------------------------------------------------------------------------------ | ----------- | -------------- |
| bart-large-mnli (baseline, 12-12) | 89.9 | 90.01 |
| distilbart-mnli-12-1 | 87.08 | 87.5 |
| distilbart-mnli-12-3 | 88.1 | 88.19 |
| distilbart-mnli-12-6 | 89.19 | 89.01 |
| distilbart-mnli-12-9 | 89.56 | 89.52 |

This is a very simple and effective technique, as we can see the performance drop is very little.

Detailed performace trade-offs will be posted in this sheet.

Fine-tuning

If you want to train these models yourself, clone the distillbart-mnli repo and follow the steps below

Clone and install transformers from source

bash
git clone https://github.com/huggingface/transformers.git
pip install -qqq -U ./transformers

Download MNLI data

bash
python transformers/utils/download_glue_data.py --data_dir glue_data --tasks MNLI

Create student model

bash
python create_student.py \
  --teacher_model_name_or_path facebook/bart-large-mnli \
  --student_encoder_layers 12 \
  --student_decoder_layers 6 \
  --save_path student-bart-mnli-12-6 \

Start fine-tuning

bash
python run_glue.py args.json

You can find the logs of these trained models in this wandb project.

查看官方来源