distilbert base uncased mnli

提供商typeform
分类zero-shot-classification
许可证Apache-2.0
下载量395.7K
星标0

简介

distilbert-base-uncased-mnli 是一款轻量级的零样本分类（Zero-shot Classification）模型。它基于 DistilBERT 蒸馏技术，在保留大部分 BERT 性能的同时显著降低了计算开销和推理延迟。该模型最核心的价值在于无需针对特定标签进行训练，即可直接对文本进行分类，非常适合需要快速验证分类效果、或者标签频繁变动的动态场景。对于开发者而言，它在 CPU 上也能流畅运行，是替代昂贵的大模型进行简单文本分类任务的理想选择。

核心亮点

  • 零样本学习，无需训练即可识别自定义标签
  • 模型轻量化，推理速度快且内存占用低
  • 支持多种语言分类任务的快速原型搭建
  • Apache-2.0 协议，商业部署无压力

使用方法

安装依赖
# 安装 Hugging Face transformers
pip install transformers torch
SDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("typeform/distilbert-base-uncased-mnli")
tokenizer = AutoTokenizer.from_pretrained("typeform/distilbert-base-uncased-mnli")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引 
pip install -U huggingface_hub

命令行下载

下载完整模型库

下载完整模型库 
huggingface-cli download typeform/distilbert-base-uncased-mnli

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例） 
huggingface-cli download typeform/distilbert-base-uncased-mnli config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

SDK 下载 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('typeform/distilbert-base-uncased-mnli')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://huggingface.co/typeform/distilbert-base-uncased-mnli

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/typeform/distilbert-base-uncased-mnli

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

安装 Transformers 
pip install -U transformers torch

模型加载和推理

模型加载和推理 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('typeform/distilbert-base-uncased-mnli')
tokenizer = AutoTokenizer.from_pretrained('typeform/distilbert-base-uncased-mnli')

完整文档

来源: HuggingFace

---
language: en
pipeline_tag: zero-shot-classification
tags:

  • distilbert

datasets:
  • multi_nli

metrics:
  • accuracy

---

DistilBERT base model (uncased)

Table of Contents

Model Details

Model Description: This is the uncased DistilBERT model fine-tuned on Multi-Genre Natural Language Inference (MNLI) dataset for the zero-shot classification task.
  • Model Type: Zero-Shot Classification
  • Language(s): English
  • License: Unknown
## How to Get Started with the Model
python
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("typeform/distilbert-base-uncased-mnli")


model = AutoModelForSequenceClassification.from_pretrained("typeform/distilbert-base-uncased-mnli")

Uses

This model can be used for text classification tasks.

Risks, Limitations and Biases

CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes.

Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)).

Training

#### Training Data

This model of DistilBERT-uncased is pretrained on the Multi-Genre Natural Language Inference (MultiNLI) corpus. It is a crowd-sourced collection of 433k sentence pairs annotated with textual entailment information. The corpus covers a range of genres of spoken and written text, and supports a distinctive cross-genre generalization evaluation.

This model is also not case-sensitive, i.e., it does not make a difference between "english" and "English".

#### Training Procedure

Training is done on a p3.2xlarge AWS EC2 with the following hyperparameters:

code
$ run_glue.py \
    --model_name_or_path distilbert-base-uncased \
    --task_name mnli \
    --do_train \
    --do_eval \
    --max_seq_length 128 \
    --per_device_train_batch_size 16 \
    --learning_rate 2e-5 \
    --num_train_epochs 5 \
    --output_dir /tmp/distilbert-base-uncased_mnli/

Evaluation

#### Evaluation Results
When fine-tuned on downstream tasks, this model achieves the following results:

  • Epoch = 5.0

  • Evaluation Accuracy = 0.8206875508543532

  • Evaluation Loss = 0.8706700205802917

  • Evaluation Runtime = 17.8278

  • Evaluation Samples per second = 551.498

MNLI and MNLI-mm results:

| Task | MNLI | MNLI-mm |
|:----:|:----:|:----:|
| | 82.0 | 82.0 |

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). We present the hardware type based on the associated paper.

Hardware Type: 1 NVIDIA Tesla V100 GPUs

Hours used: Unknown

Cloud Provider: AWS EC2 P3

Compute Region: Unknown

Carbon Emitted: (Power consumption x Time x Carbon produced based on location of power grid): Unknown

查看官方来源