xlm roberta base ner hrl

ProviderDavlan
Categorytoken-classification
Licenseafl-3.0
Downloads383.9K
Stars0

Overview

The xlm-roberta-base-ner-hrl is a specialized token-classification model built upon the XLM-RoBERTa architecture, specifically fine-tuned for Named Entity Recognition (NER). Unlike general-purpose LLMs, this model is optimized for extracting structured entities from raw text across multiple languages. It is particularly useful for developers building information extraction pipelines, automated tagging systems, or knowledge graph population tools. Because it leverages a cross-lingual backbone, it provides consistent performance across diverse language sets without requiring separate models for each locale. Integration is straightforward via the Hugging Face Transformers library, making it a lightweight, efficient alternative to calling a massive generative API for simple entity extraction tasks.

Highlights

  • Cross-lingual NER support via XLM-RoBERTa architecture
  • Optimized for high-accuracy token-level entity classification
  • Seamless integration with Hugging Face Transformers library
  • Efficient alternative to generative models for extraction
  • Permissive AFL-3.0 license for flexible deployment

Usage

Install
# Install Hugging Face transformers
pip install transformers torch
SDK Usage
# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("Davlan/xlm-roberta-base-ner-hrl")
tokenizer = AutoTokenizer.from_pretrained("Davlan/xlm-roberta-base-ner-hrl")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance 
pip install -U huggingface_hub

CLI Download

Download the full repository

Download the full repository 
huggingface-cli download Davlan/xlm-roberta-base-ner-hrl

Download a single file to a local folder (e.g. config.json into ./dir)

Download a single file to a local folder (e.g. config.json into ./dir) 
huggingface-cli download Davlan/xlm-roberta-base-ner-hrl config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

SDK Download 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('Davlan/xlm-roberta-base-ner-hrl')

Git Download

Make sure git-lfs is installed first

Git Download 
git lfs install
git clone https://huggingface.co/Davlan/xlm-roberta-base-ner-hrl

To skip LFS large-file downloads, use:

Skip LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/Davlan/xlm-roberta-base-ner-hrl

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

Install Transformers 
pip install -U transformers torch

Load the model and run inference

Load the model and run inference 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('Davlan/xlm-roberta-base-ner-hrl')
tokenizer = AutoTokenizer.from_pretrained('Davlan/xlm-roberta-base-ner-hrl')

Full Documentation

来源: HuggingFace

---
license: afl-3.0
---
Hugging Face's logo
---
language:

  • ar

  • de

  • en

  • es

  • fr

  • it

  • lv

  • nl

  • pt

  • zh

  • multilingual

---

xlm-roberta-base-ner-hrl


Model description


xlm-roberta-base-ner-hrl is a Named Entity Recognition model for 10 high resourced languages (Arabic, German, English, Spanish, French, Italian, Latvian, Dutch, Portuguese and Chinese) based on a fine-tuned XLM-RoBERTa base model. It has been trained to recognize three types of entities: location (LOC), organizations (ORG), and person (PER).
Specifically, this model is a *xlm-roberta-base* model that was fine-tuned on an aggregation of 10 high-resourced languages

Intended uses & limitations


#### How to use
You can use this model with Transformers *pipeline* for NER.
python
from transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import pipeline
tokenizer = AutoTokenizer.from_pretrained("Davlan/xlm-roberta-base-ner-hrl")
model = AutoModelForTokenClassification.from_pretrained("Davlan/xlm-roberta-base-ner-hrl")
nlp = pipeline("ner", model=model, tokenizer=tokenizer)
example = "Nader Jokhadar had given Syria the lead with a well-struck header in the seventh minute."
ner_results = nlp(example)
print(ner_results)

#### Limitations and bias
This model is limited by its training dataset of entity-annotated news articles from a specific span of time. This may not generalize well for all use cases in different domains.

Training data


The training data for the 10 languages are from:

Language|Dataset
-|-
Arabic | ANERcorp
German | conll 2003
English | conll 2003
Spanish | conll 2002
French | Europeana Newspapers
Italian | Italian I-CAB
Latvian | Latvian NER
Dutch | conll 2002
Portuguese |Paramopama + Second Harem
Chinese | MSRA

The training dataset distinguishes between the beginning and continuation of an entity so that if there are back-to-back entities of the same type, the model can output where the second entity begins. As in the dataset, each token will be classified as one of the following classes:
Abbreviation|Description
-|-
O|Outside of a named entity
B-PER |Beginning of a person’s name right after another person’s name
I-PER |Person’s name
B-ORG |Beginning of an organisation right after another organisation
I-ORG |Organisation
B-LOC |Beginning of a location right after another location
I-LOC |Location

Training procedure


This model was trained on NVIDIA V100 GPU with recommended hyperparameters from HuggingFace code.

View official source
Join our Telegram