DeBERTa v3 large mnli fever anli ling wanli
简介
这是一个基于 DeBERTa v3 large 架构并经过多数据集（如 MNLI、FEVER 等）强化训练的零样本分类模型。它最大的特点是无需针对特定标签进行微调，就能直接对文本进行分类。对于开发者来说，它解决了小样本场景下标注成本高的问题，非常适合用于快速搭建文本打标签系统、意图识别或内容审核原型。相比于通用大模型，它轻量且推理速度快，在特定分类任务上的准确度极具竞争力，是替代复杂 Prompt 工程的理想选择。
核心亮点
- 无需训练数据即可实现零样本文本分类
- 多数据集强化训练，逻辑推理能力更强
- 推理开销远低于 LLM，适合生产环境部署
- 极速构建意图识别与内容打标工作流
使用方法
安装依赖
# 安装 Hugging Face transformers
pip install transformers torchSDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli")
tokenizer = AutoTokenizer.from_pretrained("MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli")Hugging Face 下载
我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。
操作指引：在下载前，请先通过如下命令安装 huggingface_hub：
操作指引
pip install -U huggingface_hub命令行下载
下载完整模型库
下载完整模型库
huggingface-cli download MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）
下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）
huggingface-cli download MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli config.json --local-dir ./dirSDK 下载
SDK 下载
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli')Git 下载
请确保 lfs 已经被正确安装
Git 下载
git lfs install
git clone https://huggingface.co/MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli如果您希望跳过 lfs 大文件下载，可以使用如下命令
跳过 LFS
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。
PyTorch / Transformers 使用
安装 Transformers
安装 Transformers
pip install -U transformers torch模型加载和推理
模型加载和推理
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained('MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli')
tokenizer = AutoTokenizer.from_pretrained('MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli')完整文档
来源: HuggingFace
---
language:
- en
license: mit
tags:
- text-classification
- zero-shot-classification
datasets:
- multi_nli
- facebook/anli
- fever
- lingnli
- alisawuffles/WANLI
metrics:
- accuracy
pipeline_tag: zero-shot-classification
model-index:
- name: DeBERTa-v3-large-mnli-fever-anli-ling-wanli
results:
- task:
type: text-classification
name: Natural Language Inference
dataset:
name: MultiNLI-matched
type: multi_nli
split: validation_matched
metrics:
- type: accuracy
value: 0,912
verified: false
- task:
type: text-classification
name: Natural Language Inference
dataset:
name: MultiNLI-mismatched
type: multi_nli
split: validation_mismatched
metrics:
- type: accuracy
value: 0,908
verified: false
- task:
type: text-classification
name: Natural Language Inference
dataset:
name: ANLI-all
type: anli
split: test_r1+test_r2+test_r3
metrics:
- type: accuracy
value: 0,702
verified: false
- task:
type: text-classification
name: Natural Language Inference
dataset:
name: ANLI-r3
type: anli
split: test_r3
metrics:
- type: accuracy
value: 0,64
verified: false
- task:
type: text-classification
name: Natural Language Inference
dataset:
name: WANLI
type: alisawuffles/WANLI
split: test
metrics:
- type: accuracy
value: 0,77
verified: false
- task:
type: text-classification
name: Natural Language Inference
dataset:
name: LingNLI
type: lingnli
split: test
metrics:
- type: accuracy
value: 0,87
verified: false
---
DeBERTa-v3-large-mnli-fever-anli-ling-wanli
Model description
This model was fine-tuned on the MultiNLI, Fever-NLI, Adversarial-NLI (ANLI), LingNLI and WANLI datasets, which comprise 885 242 NLI hypothesis-premise pairs. This model is the best performing NLI model on the Hugging Face Hub as of 06.06.22 and can be used for zero-shot classification. It significantly outperforms all other large models on the ANLI benchmark.The foundation model is DeBERTa-v3-large from Microsoft. DeBERTa-v3 combines several recent innovations compared to classical Masked Language Models like BERT, RoBERTa etc., see the paper
How to use the model
#### Simple zero-shot classification pipelinepython
from transformers import pipeline
classifier = pipeline("zero-shot-classification", model="MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli")
sequence_to_classify = "Angela Merkel is a politician in Germany and leader of the CDU"
candidate_labels = ["politics", "economy", "entertainment", "environment"]
output = classifier(sequence_to_classify, candidate_labels, multi_label=False)
print(output)python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
model_name = "MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
premise = "I first thought that I liked the movie, but upon second thought it was actually disappointing."
hypothesis = "The movie was not good."
input = tokenizer(premise, hypothesis, truncation=True, return_tensors="pt")
output = model(input["input_ids"].to(device)) # device = "cuda:0" or "cpu"
prediction = torch.softmax(output["logits"][0], -1).tolist()
label_names = ["entailment", "neutral", "contradiction"]
prediction = {name: round(float(pred) * 100, 1) for pred, name in zip(prediction, label_names)}
print(prediction)
Training data
DeBERTa-v3-large-mnli-fever-anli-ling-wanli was trained on the MultiNLI, Fever-NLI, Adversarial-NLI (ANLI), LingNLI and WANLI datasets, which comprise 885 242 NLI hypothesis-premise pairs. Note that SNLI was explicitly excluded due to quality issues with the dataset. More data does not necessarily make for better NLI models.Training procedure
DeBERTa-v3-large-mnli-fever-anli-ling-wanli was trained using the Hugging Face trainer with the following hyperparameters. Note that longer training with more epochs hurt performance in my tests (overfitting).code
training_args = TrainingArguments(
num_train_epochs=4, # total number of training epochs
learning_rate=5e-06,
per_device_train_batch_size=16, # batch size per device during training
gradient_accumulation_steps=2, # doubles the effective batch_size to 32, while decreasing memory requirements
per_device_eval_batch_size=64, # batch size for evaluation
warmup_ratio=0.06, # number of warmup steps for learning rate scheduler
weight_decay=0.01, # strength of weight decay
fp16=True # mixed precision training
)Eval results
The model was evaluated using the test sets for MultiNLI, ANLI, LingNLI, WANLI and the dev set for Fever-NLI. The metric used is accuracy. The model achieves state-of-the-art performance on each dataset. Surprisingly, it outperforms the previous state-of-the-art on ANLI (ALBERT-XXL) by 8,3%. I assume that this is because ANLI was created to fool masked language models like RoBERTa (or ALBERT), while DeBERTa-v3 uses a better pre-training objective (RTD), disentangled attention and I fine-tuned it on higher quality NLI data.|Datasets|mnli_test_m|mnli_test_mm|anli_test|anli_test_r3|ling_test|wanli_test|
| :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|Accuracy|0.912|0.908|0.702|0.64|0.87|0.77|
|Speed (text/sec, A100 GPU)|696.0|697.0|488.0|425.0|828.0|980.0|
