mobilebert uncased squad v2

Providercsarron
Categoryquestion-answering
Licensemit
Downloads21.3K
Stars0

Overview

MobileBERT Uncased SQuAD v2 is a streamlined transformer model optimized for resource-constrained environments without sacrificing significant accuracy. Unlike standard BERT, it utilizes a bottleneck structure to reduce parameter count and latency, making it ideal for on-device deployment on mobile or edge hardware. This specific version is fine-tuned on SQuAD v2.0, meaning it can handle 'unanswerable' questions—a critical feature for production QA systems to prevent hallucinations. For developers, this offers a pragmatic middle ground: you get the extractive question-answering capabilities of a large language model with a footprint that fits into limited RAM and executes faster on CPUs.

Highlights

  • Optimized for low-latency on-device inference
  • Fine-tuned for SQuAD v2 extractive QA
  • Handles unanswerable queries to reduce hallucinations
  • Significant parameter reduction compared to BERT-base
  • MIT licensed for flexible commercial integration

Usage

Install
# Install Hugging Face transformers
pip install transformers torch
SDK Usage
# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("csarron/mobilebert-uncased-squad-v2")
tokenizer = AutoTokenizer.from_pretrained("csarron/mobilebert-uncased-squad-v2")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance 
pip install -U huggingface_hub

CLI Download

Download the full repository

Download the full repository 
huggingface-cli download csarron/mobilebert-uncased-squad-v2

Download a single file to a local folder (e.g. config.json into ./dir)

Download a single file to a local folder (e.g. config.json into ./dir) 
huggingface-cli download csarron/mobilebert-uncased-squad-v2 config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

SDK Download 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('csarron/mobilebert-uncased-squad-v2')

Git Download

Make sure git-lfs is installed first

Git Download 
git lfs install
git clone https://huggingface.co/csarron/mobilebert-uncased-squad-v2

To skip LFS large-file downloads, use:

Skip LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/csarron/mobilebert-uncased-squad-v2

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

Install Transformers 
pip install -U transformers torch

Load the model and run inference

Load the model and run inference 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('csarron/mobilebert-uncased-squad-v2')
tokenizer = AutoTokenizer.from_pretrained('csarron/mobilebert-uncased-squad-v2')

Full Documentation

来源: HuggingFace

---
language: en
thumbnail:
license: mit
tags:

  • question-answering

  • mobilebert

datasets:
  • squad_v2

metrics:
  • squad_v2

widget:
  • text: "Which name is also used to describe the Amazon rainforest in English?"

context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species."
  • text: "How many square kilometers of rainforest is covered in the basin?"

context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species."
---

MobileBERT fine-tuned on SQuAD v2

MobileBERT is a thin version of BERT_LARGE, while equipped with bottleneck structures and a carefully designed balance
between self-attentions and feed-forward networks.

This model was fine-tuned from the HuggingFace checkpoint google/mobilebert-uncased on SQuAD2.0.

Details

| Dataset | Split | # samples |
| -------- | ----- | --------- |
| SQuAD2.0 | train | 130k |
| SQuAD2.0 | eval | 12.3k |

Fine-tuning

  • Python: 3.7.5
  • Machine specs:

CPU: Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz

Memory: 32 GiB

GPUs: 2 GeForce GTX 1070, each with 8GiB memory

GPU driver: 418.87.01, CUDA: 10.1

  • script:
shell
# after install https://github.com/huggingface/transformers

cd examples/question-answering
  mkdir -p data


wget -O data/train-v2.0.json https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v2.0.json


wget -O data/dev-v2.0.json  https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v2.0.json


export SQUAD_DIR=pwd/data


python run_squad.py \
    --model_type mobilebert \
    --model_name_or_path google/mobilebert-uncased \
    --do_train \
    --do_eval \
    --do_lower_case \
    --version_2_with_negative \
    --train_file $SQUAD_DIR/train-v2.0.json \
    --predict_file $SQUAD_DIR/dev-v2.0.json \
    --per_gpu_train_batch_size 16 \
    --per_gpu_eval_batch_size 16 \
    --learning_rate 4e-5 \
    --num_train_epochs 5.0 \
    --max_seq_length 320 \
    --doc_stride 128 \
    --warmup_steps 1400 \
    --save_steps 2000 \
    --output_dir $SQUAD_DIR/mobilebert-uncased-warmup-squad_v2 2>&1 | tee train-mobilebert-warmup-squad_v2.log

It took about 3.5 hours to finish.

Results

Model size: 95M

| Metric | # Value | # Original (Table 5)|
| ------ | --------- | --------- |
| EM | 75.2 | 76.2 |
| F1 | 78.8 | 79.2 |

Note that the above results didn't involve any hyperparameter search.

Example Usage

python
from transformers import pipeline

qa_pipeline = pipeline(
    "question-answering",
    model="csarron/mobilebert-uncased-squad-v2",
    tokenizer="csarron/mobilebert-uncased-squad-v2"
)


predictions = qa_pipeline({
    'context': "The game was played on February 7, 2016 at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California.",
    'question': "What day was the game played on?"
})


print(predictions)
output:

{'score': 0.71434086561203, 'start': 23, 'end': 39, 'answer': 'February 7, 2016'}

> Created by Qingqing Cao | GitHub | Twitter

> Made with ❤️ in New York.

View official source
Join our Telegram