mobilebert uncased squad v2

提供商csarron
分类question-answering
许可证mit
下载量21.3K
星标0

简介

MobileBERT Uncased SQuAD v2 是一款专为移动端和边缘设备优化的轻量级问答模型。它通过知识蒸馏技术，在保持接近 BERT 性能的同时，大幅降低了模型参数量和计算延迟。该模型在 SQuAD v2 数据集上进行了微调，能够处理包含“不可回答问题”的复杂问答场景。对于需要将 QA 功能集成到 App 或嵌入式设备，且无法依赖高昂 GPU 云端推理的开发者来说，这是一个极佳的端侧部署选择，上手难度低，兼容 Hugging Face 生态。

核心亮点

  • 轻量化设计，极速适配移动端与边缘设备
  • 支持 SQuAD v2，能识别无法回答的陷阱问题
  • 低延迟推理，显著降低端侧硬件资源占用
  • 兼容 Transformers 库，部署迁移成本极低

使用方法

安装依赖
# 安装 Hugging Face transformers
pip install transformers torch
SDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("csarron/mobilebert-uncased-squad-v2")
tokenizer = AutoTokenizer.from_pretrained("csarron/mobilebert-uncased-squad-v2")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引 
pip install -U huggingface_hub

命令行下载

下载完整模型库

下载完整模型库 
huggingface-cli download csarron/mobilebert-uncased-squad-v2

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例） 
huggingface-cli download csarron/mobilebert-uncased-squad-v2 config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

SDK 下载 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('csarron/mobilebert-uncased-squad-v2')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://huggingface.co/csarron/mobilebert-uncased-squad-v2

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/csarron/mobilebert-uncased-squad-v2

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

安装 Transformers 
pip install -U transformers torch

模型加载和推理

模型加载和推理 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('csarron/mobilebert-uncased-squad-v2')
tokenizer = AutoTokenizer.from_pretrained('csarron/mobilebert-uncased-squad-v2')

完整文档

来源: HuggingFace

---
language: en
thumbnail:
license: mit
tags:

  • question-answering

  • mobilebert

datasets:
  • squad_v2

metrics:
  • squad_v2

widget:
  • text: "Which name is also used to describe the Amazon rainforest in English?"

context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species."
  • text: "How many square kilometers of rainforest is covered in the basin?"

context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species."
---

MobileBERT fine-tuned on SQuAD v2

MobileBERT is a thin version of BERT_LARGE, while equipped with bottleneck structures and a carefully designed balance
between self-attentions and feed-forward networks.

This model was fine-tuned from the HuggingFace checkpoint google/mobilebert-uncased on SQuAD2.0.

Details

| Dataset | Split | # samples |
| -------- | ----- | --------- |
| SQuAD2.0 | train | 130k |
| SQuAD2.0 | eval | 12.3k |

Fine-tuning

  • Python: 3.7.5
  • Machine specs:

CPU: Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz

Memory: 32 GiB

GPUs: 2 GeForce GTX 1070, each with 8GiB memory

GPU driver: 418.87.01, CUDA: 10.1

  • script:
shell
# after install https://github.com/huggingface/transformers

cd examples/question-answering
  mkdir -p data


wget -O data/train-v2.0.json https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v2.0.json


wget -O data/dev-v2.0.json  https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v2.0.json


export SQUAD_DIR=pwd/data


python run_squad.py \
    --model_type mobilebert \
    --model_name_or_path google/mobilebert-uncased \
    --do_train \
    --do_eval \
    --do_lower_case \
    --version_2_with_negative \
    --train_file $SQUAD_DIR/train-v2.0.json \
    --predict_file $SQUAD_DIR/dev-v2.0.json \
    --per_gpu_train_batch_size 16 \
    --per_gpu_eval_batch_size 16 \
    --learning_rate 4e-5 \
    --num_train_epochs 5.0 \
    --max_seq_length 320 \
    --doc_stride 128 \
    --warmup_steps 1400 \
    --save_steps 2000 \
    --output_dir $SQUAD_DIR/mobilebert-uncased-warmup-squad_v2 2>&1 | tee train-mobilebert-warmup-squad_v2.log

It took about 3.5 hours to finish.

Results

Model size: 95M

| Metric | # Value | # Original (Table 5)|
| ------ | --------- | --------- |
| EM | 75.2 | 76.2 |
| F1 | 78.8 | 79.2 |

Note that the above results didn't involve any hyperparameter search.

Example Usage

python
from transformers import pipeline

qa_pipeline = pipeline(
    "question-answering",
    model="csarron/mobilebert-uncased-squad-v2",
    tokenizer="csarron/mobilebert-uncased-squad-v2"
)


predictions = qa_pipeline({
    'context': "The game was played on February 7, 2016 at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California.",
    'question': "What day was the game played on?"
})


print(predictions)
output:

{'score': 0.71434086561203, 'start': 23, 'end': 39, 'answer': 'February 7, 2016'}

> Created by Qingqing Cao | GitHub | Twitter

> Made with ❤️ in New York.

查看官方来源