mdeberta v3 base squad2

提供商timpal0l
分类question-answering
许可证mit
下载量67.4K
星标0

简介

mdeberta-v3-base-squad2 是一款基于微软 DeBERTa-v3 架构并经过 SQuAD 2.0 数据集微调的阅读理解模型。相比传统的 BERT，它在处理语义理解和上下文关联上更强，尤其擅长从给定的文本段落中精准提取答案。该模型支持“不可回答”问题的识别，能有效减少 AI 强行回答导致的幻觉问题。对于开发者而言，它是一个轻量且高效的抽取式 QA 方案，非常适合集成到本地知识库、智能客服或文档分析工具中，上手门槛低，推理速度快。

核心亮点

  • 基于 DeBERTa-v3，语义理解能力优于 BERT
  • 专为 SQuAD 2.0 优化，支持识别不可回答问题
  • 典型的抽取式 QA 模型，答案定位精准
  • 模型规模适中，适合本地部署和快速推理

使用方法

安装依赖
# 安装 Hugging Face transformers
pip install transformers torch
SDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("timpal0l/mdeberta-v3-base-squad2")
tokenizer = AutoTokenizer.from_pretrained("timpal0l/mdeberta-v3-base-squad2")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引 
pip install -U huggingface_hub

命令行下载

下载完整模型库

下载完整模型库 
huggingface-cli download timpal0l/mdeberta-v3-base-squad2

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例） 
huggingface-cli download timpal0l/mdeberta-v3-base-squad2 config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

SDK 下载 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('timpal0l/mdeberta-v3-base-squad2')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://huggingface.co/timpal0l/mdeberta-v3-base-squad2

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/timpal0l/mdeberta-v3-base-squad2

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

安装 Transformers 
pip install -U transformers torch

模型加载和推理

模型加载和推理 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('timpal0l/mdeberta-v3-base-squad2')
tokenizer = AutoTokenizer.from_pretrained('timpal0l/mdeberta-v3-base-squad2')

完整文档

来源: HuggingFace

---
datasets:

  • squad_v2

language:
  • multilingual

  • af

  • am

  • ar

  • as

  • az

  • be

  • bg

  • bn

  • br

  • bs

  • ca

  • cs

  • cy

  • da

  • de

  • el

  • en

  • eo

  • es

  • et

  • eu

  • fa

  • fi

  • fr

  • fy

  • ga

  • gd

  • gl

  • gu

  • ha

  • he

  • hi

  • hr

  • hu

  • hy

  • id

  • is

  • it

  • ja

  • jv

  • ka

  • kk

  • km

  • kn

  • ko

  • ku

  • ky

  • la

  • lo

  • lt

  • lv

  • mg

  • mk

  • ml

  • mn

  • mr

  • ms

  • my

  • ne

  • nl

  • 'no'

  • om

  • or

  • pa

  • pl

  • ps

  • pt

  • ro

  • ru

  • sa

  • sd

  • si

  • sk

  • sl

  • so

  • sq

  • sr

  • su

  • sv

  • sw

  • ta

  • te

  • th

  • tl

  • tr

  • ug

  • uk

  • ur

  • uz

  • vi

  • xh

  • yi

  • zh

tags:
  • deberta

  • deberta-v3

  • mdeberta

  • question-answering

  • qa

  • multilingual

thumbnail: https://huggingface.co/front/thumbnails/microsoft.png
license: mit
base_model:
  • microsoft/mdeberta-v3-base

---

This model can be used for Extractive QA


It has been finetuned for 3 epochs on SQuAD2.0.

Usage

python
from transformers import pipeline

qa_model = pipeline("question-answering", "timpal0l/mdeberta-v3-base-squad2")
question = "Where do I live?"
context = "My name is Tim and I live in Sweden."
qa_model(question = question, context = context)
{'score': 0.975547730922699, 'start': 28, 'end': 36, 'answer': ' Sweden.'}

Evaluation on SQuAD2.0 dev set

bash
{
    "epoch": 3.0,
    "eval_HasAns_exact": 79.65587044534414,
    "eval_HasAns_f1": 85.91387795001529,
    "eval_HasAns_total": 5928,
    "eval_NoAns_exact": 82.10260723296888,
    "eval_NoAns_f1": 82.10260723296888,
    "eval_NoAns_total": 5945,
    "eval_best_exact": 80.8809904826076,
    "eval_best_exact_thresh": 0.0,
    "eval_best_f1": 84.00551406448994,
    "eval_best_f1_thresh": 0.0,
    "eval_exact": 80.8809904826076,
    "eval_f1": 84.00551406449004,
    "eval_samples": 12508,
    "eval_total": 11873,
    "train_loss": 0.7729689576483615,
    "train_runtime": 9118.953,
    "train_samples": 134891,
    "train_samples_per_second": 44.377,
    "train_steps_per_second": 0.925
}

DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing

DeBERTa improves the BERT and RoBERTa models using disentangled attention and enhanced mask decoder. With those two improvements, DeBERTa out perform RoBERTa on a majority of NLU tasks with 80GB training data.

In DeBERTa V3, we further improved the efficiency of DeBERTa using ELECTRA-Style pre-training with Gradient Disentangled Embedding Sharing. Compared to DeBERTa, our V3 version significantly improves the model performance on downstream tasks. You can find more technique details about the new model from our paper.

Please check the official repository for more implementation details and updates.

mDeBERTa is multilingual version of DeBERTa which use the same structure as DeBERTa and was trained with CC100 multilingual data.
The mDeBERTa V3 base model comes with 12 layers and a hidden size of 768. It has 86M backbone parameters with a vocabulary containing 250K tokens which introduces 190M parameters in the Embedding layer. This model was trained using the 2.5T CC100 data as XLM-R.

查看官方来源