distilbert base uncased mnli

Providertypeform
Categoryzero-shot-classification
LicenseApache-2.0
Downloads395.7K
Stars0

Overview

DistilBERT base uncased MNLI is a streamlined transformer model optimized for zero-shot text classification. By leveraging a knowledge distillation process from BERT, it provides a high-performance alternative for developers who need to categorize text into arbitrary labels without the overhead of training a task-specific classifier. It is particularly effective for real-time sentiment analysis, intent detection, and dynamic tagging where labels may change frequently. Integration is straightforward via standard Hugging Face pipelines, offering a significant reduction in latency and memory footprint compared to full BERT models while retaining most of the NLI (Natural Language Inference) capabilities required for zero-shot accuracy.

Highlights

  • Fast zero-shot classification without task-specific training
  • Reduced memory footprint via BERT knowledge distillation
  • Apache-2.0 license for flexible commercial integration
  • Low-latency inference ideal for real-time production apps
  • Seamless deployment through standard transformer libraries

Usage

Install
# Install Hugging Face transformers
pip install transformers torch
SDK Usage
# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("typeform/distilbert-base-uncased-mnli")
tokenizer = AutoTokenizer.from_pretrained("typeform/distilbert-base-uncased-mnli")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance 
pip install -U huggingface_hub

CLI Download

Download the full repository

Download the full repository 
huggingface-cli download typeform/distilbert-base-uncased-mnli

Download a single file to a local folder (e.g. config.json into ./dir)

Download a single file to a local folder (e.g. config.json into ./dir) 
huggingface-cli download typeform/distilbert-base-uncased-mnli config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

SDK Download 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('typeform/distilbert-base-uncased-mnli')

Git Download

Make sure git-lfs is installed first

Git Download 
git lfs install
git clone https://huggingface.co/typeform/distilbert-base-uncased-mnli

To skip LFS large-file downloads, use:

Skip LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/typeform/distilbert-base-uncased-mnli

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

Install Transformers 
pip install -U transformers torch

Load the model and run inference

Load the model and run inference 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('typeform/distilbert-base-uncased-mnli')
tokenizer = AutoTokenizer.from_pretrained('typeform/distilbert-base-uncased-mnli')

Full Documentation

来源: HuggingFace

---
language: en
pipeline_tag: zero-shot-classification
tags:

  • distilbert

datasets:
  • multi_nli

metrics:
  • accuracy

---

DistilBERT base model (uncased)

Table of Contents

Model Details

Model Description: This is the uncased DistilBERT model fine-tuned on Multi-Genre Natural Language Inference (MNLI) dataset for the zero-shot classification task.
  • Model Type: Zero-Shot Classification
  • Language(s): English
  • License: Unknown
## How to Get Started with the Model
python
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("typeform/distilbert-base-uncased-mnli")


model = AutoModelForSequenceClassification.from_pretrained("typeform/distilbert-base-uncased-mnli")

Uses

This model can be used for text classification tasks.

Risks, Limitations and Biases

CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes.

Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)).

Training

#### Training Data

This model of DistilBERT-uncased is pretrained on the Multi-Genre Natural Language Inference (MultiNLI) corpus. It is a crowd-sourced collection of 433k sentence pairs annotated with textual entailment information. The corpus covers a range of genres of spoken and written text, and supports a distinctive cross-genre generalization evaluation.

This model is also not case-sensitive, i.e., it does not make a difference between "english" and "English".

#### Training Procedure

Training is done on a p3.2xlarge AWS EC2 with the following hyperparameters:

code
$ run_glue.py \
    --model_name_or_path distilbert-base-uncased \
    --task_name mnli \
    --do_train \
    --do_eval \
    --max_seq_length 128 \
    --per_device_train_batch_size 16 \
    --learning_rate 2e-5 \
    --num_train_epochs 5 \
    --output_dir /tmp/distilbert-base-uncased_mnli/

Evaluation

#### Evaluation Results
When fine-tuned on downstream tasks, this model achieves the following results:

  • Epoch = 5.0

  • Evaluation Accuracy = 0.8206875508543532

  • Evaluation Loss = 0.8706700205802917

  • Evaluation Runtime = 17.8278

  • Evaluation Samples per second = 551.498

MNLI and MNLI-mm results:

| Task | MNLI | MNLI-mm |
|:----:|:----:|:----:|
| | 82.0 | 82.0 |

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). We present the hardware type based on the associated paper.

Hardware Type: 1 NVIDIA Tesla V100 GPUs

Hours used: Unknown

Cloud Provider: AWS EC2 P3

Compute Region: Unknown

Carbon Emitted: (Power consumption x Time x Carbon produced based on location of power grid): Unknown

View official source
Join our Telegram