Name: deberta v3 xsmall zeroshot v1.1 all 33
Author: MoritzLaurer

Overview

DeBERTa-v3-xsmall-zeroshot-v1.1 is a lightweight, high-efficiency model designed for zero-shot text classification. Unlike traditional classifiers that require labeled training data for every new category, this model leverages a Natural Language Inference (NLI) approach to categorize text into arbitrary labels on the fly. For developers, this means immediate deployment for dynamic tagging or content routing without the overhead of custom fine-tuning. While it lacks the raw power of larger LLMs, its 'xsmall' architecture ensures low latency and minimal VRAM usage, making it ideal for edge deployment or high-throughput preprocessing pipelines where cost-per-inference is a critical metric.

Highlights

Zero-shot classification without requiring labeled training data
Low-latency inference ideal for edge and real-time applications
Based on efficient DeBERTa-v3 architecture for better accuracy
MIT licensed for flexible commercial and private integration

Usage

Install

# Install Hugging Face transformers
pip install transformers torch

SDK Usage

# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("MoritzLaurer/deberta-v3-xsmall-zeroshot-v1.1-all-33")
tokenizer = AutoTokenizer.from_pretrained("MoritzLaurer/deberta-v3-xsmall-zeroshot-v1.1-all-33")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance

pip install -U huggingface_hub

CLI Download

Download the full repository

huggingface-cli download MoritzLaurer/deberta-v3-xsmall-zeroshot-v1.1-all-33

Download a single file to a local folder (e.g. config.json into ./dir)

huggingface-cli download MoritzLaurer/deberta-v3-xsmall-zeroshot-v1.1-all-33 config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('MoritzLaurer/deberta-v3-xsmall-zeroshot-v1.1-all-33')

Git Download

Make sure git-lfs is installed first

Git Download

git lfs install
git clone https://huggingface.co/MoritzLaurer/deberta-v3-xsmall-zeroshot-v1.1-all-33

To skip LFS large-file downloads, use:

Skip LFS

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/MoritzLaurer/deberta-v3-xsmall-zeroshot-v1.1-all-33

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

pip install -U transformers torch

Load the model and run inference

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('MoritzLaurer/deberta-v3-xsmall-zeroshot-v1.1-all-33')
tokenizer = AutoTokenizer.from_pretrained('MoritzLaurer/deberta-v3-xsmall-zeroshot-v1.1-all-33')

Full Documentation

来源: HuggingFace

---
base_model: microsoft/deberta-v3-xsmall
language:

en

tags:

text-classification

zero-shot-classification

pipeline_tag: zero-shot-classification
library_name: transformers
license: mit
---

deberta-v3-xsmall-zeroshot-v1.1-all-33

This model was fine-tuned using the same pipeline as described in
the model card for MoritzLaurer/deberta-v3-large-zeroshot-v1.1-all-33
and in this paper.

The foundation model is microsoft/deberta-v3-xsmall.
The model only has 22 million backbone parameters and 128 million vocabulary parameters.
The backbone parameters are the main parameters active during inference, providing a significant speedup over larger models.
The model is 142 MB small.

This model was trained to provide a small and highly efficient zeroshot option,
especially for edge devices or in-browser use-cases with transformers.js.

Usage and other details

For usage instructions and other details refer to this model card MoritzLaurer/deberta-v3-large-zeroshot-v1.1-all-33 and this paper.

Metrics:

I didn't not do zeroshot evaluation for this model to save time and compute.
The table below shows standard accuracy for all datasets the model was trained on (note that the NLI datasets are binary).

General takeaway: the model is much more efficient than its larger sisters, but it performs less well.

|Datasets|mnli_m|mnli_mm|fevernli|anli_r1|anli_r2|anli_r3|wanli|lingnli|wellformedquery|rottentomatoes|amazonpolarity|imdb|yelpreviews|hatexplain|massive|banking77|emotiondair|emocontext|empathetic|agnews|yahootopics|biasframes_sex|biasframes_offensive|biasframes_intent|financialphrasebank|appreviews|hateoffensive|trueteacher|spam|wikitoxic_toxicaggregated|wikitoxic_obscene|wikitoxic_identityhate|wikitoxic_threat|wikitoxic_insult|manifesto|capsotu|
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|Accuracy|0.925|0.923|0.886|0.732|0.633|0.661|0.814|0.887|0.722|0.872|0.944|0.925|0.967|0.774|0.734|0.627|0.762|0.745|0.465|0.888|0.702|0.94|0.853|0.863|0.914|0.926|0.921|0.635|0.968|0.897|0.918|0.915|0.935|0.9|0.505|0.701|
|Inference text/sec (A10G, batch=128)|1573.0|1630.0|683.0|1282.0|1352.0|1072.0|2325.0|2008.0|4781.0|2743.0|677.0|228.0|238.0|2357.0|5027.0|4323.0|3247.0|3129.0|941.0|1643.0|335.0|1517.0|1452.0|1498.0|2367.0|974.0|2634.0|353.0|2284.0|260.0|252.0|256.0|254.0|259.0|1941.0|2080.0|