bert base multilingual cased ner hrl

ProviderDavlan
Categorytoken-classification
Licenseafl-3.0
Downloads301.9K
Stars0

Overview

The bert-base-multilingual-cased-ner-hrl is a specialized token-classification model designed for Named Entity Recognition (NER) across multiple languages. Built upon the multilingual BERT architecture, it excels at identifying and categorizing entities within diverse linguistic contexts while preserving case sensitivity, which is critical for accurate proper noun detection. For developers, this model provides a reliable alternative to training custom NER pipelines from scratch, particularly for cross-lingual applications where consistent entity extraction is required across different regional datasets. It integrates seamlessly into standard Hugging Face transformers pipelines, making it straightforward to deploy in production environments for automated data labeling, knowledge graph construction, or information retrieval systems.

Highlights

  • Multilingual support for cross-border entity extraction
  • Case-sensitive token classification for higher precision
  • Seamless integration with Hugging Face transformers
  • Optimized for diverse NER production use cases

Usage

Install
# Install Hugging Face transformers
pip install transformers torch
SDK Usage
# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("Davlan/bert-base-multilingual-cased-ner-hrl")
tokenizer = AutoTokenizer.from_pretrained("Davlan/bert-base-multilingual-cased-ner-hrl")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance 
pip install -U huggingface_hub

CLI Download

Download the full repository

Download the full repository 
huggingface-cli download Davlan/bert-base-multilingual-cased-ner-hrl

Download a single file to a local folder (e.g. config.json into ./dir)

Download a single file to a local folder (e.g. config.json into ./dir) 
huggingface-cli download Davlan/bert-base-multilingual-cased-ner-hrl config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

SDK Download 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('Davlan/bert-base-multilingual-cased-ner-hrl')

Git Download

Make sure git-lfs is installed first

Git Download 
git lfs install
git clone https://huggingface.co/Davlan/bert-base-multilingual-cased-ner-hrl

To skip LFS large-file downloads, use:

Skip LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/Davlan/bert-base-multilingual-cased-ner-hrl

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

Install Transformers 
pip install -U transformers torch

Load the model and run inference

Load the model and run inference 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('Davlan/bert-base-multilingual-cased-ner-hrl')
tokenizer = AutoTokenizer.from_pretrained('Davlan/bert-base-multilingual-cased-ner-hrl')

Full Documentation

来源: HuggingFace

---
license: afl-3.0
---
Hugging Face's logo
---
language:

  • ar

  • de

  • en

  • es

  • fr

  • it

  • lv

  • nl

  • pt

  • zh

  • multilingual

---

bert-base-multilingual-cased-ner-hrl


Model description


bert-base-multilingual-cased-ner-hrl is a Named Entity Recognition model for 10 high resourced languages (Arabic, German, English, Spanish, French, Italian, Latvian, Dutch, Portuguese and Chinese) based on a fine-tuned mBERT base model. It has been trained to recognize three types of entities: location (LOC), organizations (ORG), and person (PER).
Specifically, this model is a *bert-base-multilingual-cased* model that was fine-tuned on an aggregation of 10 high-resourced languages

Intended uses & limitations


#### How to use
You can use this model with Transformers *pipeline* for NER.
python
from transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import pipeline
tokenizer = AutoTokenizer.from_pretrained("Davlan/bert-base-multilingual-cased-ner-hrl")
model = AutoModelForTokenClassification.from_pretrained("Davlan/bert-base-multilingual-cased-ner-hrl")
nlp = pipeline("ner", model=model, tokenizer=tokenizer)
example = "Nader Jokhadar had given Syria the lead with a well-struck header in the seventh minute."
ner_results = nlp(example)
print(ner_results)

#### Limitations and bias
This model is limited by its training dataset of entity-annotated news articles from a specific span of time. This may not generalize well for all use cases in different domains.

Training data


The training data for the 10 languages are from:

Language|Dataset
-|-
Arabic | ANERcorp
German | conll 2003
English | conll 2003
Spanish | conll 2002
French | Europeana Newspapers
Italian | Italian I-CAB
Latvian | Latvian NER
Dutch | conll 2002
Portuguese |Paramopama + Second Harem
Chinese | MSRA

The training dataset distinguishes between the beginning and continuation of an entity so that if there are back-to-back entities of the same type, the model can output where the second entity begins. As in the dataset, each token will be classified as one of the following classes:
Abbreviation|Description
-|-
O|Outside of a named entity
B-PER |Beginning of a person’s name right after another person’s name
I-PER |Person’s name
B-ORG |Beginning of an organisation right after another organisation
I-ORG |Organisation
B-LOC |Beginning of a location right after another location
I-LOC |Location

Training procedure


This model was trained on NVIDIA V100 GPU with recommended hyperparameters from HuggingFace code.

View official source
Join our Telegram