ner english fast

提供商flair
分类token-classification
许可证Apache-2.0
下载量91
星标0

简介

ner-english-fast 是由 Flair 团队推出的轻量级英文命名实体识别（NER）模型。它专为追求推理速度的场景设计，能够快速从英文文本中提取人名、地名和组织机构等关键实体。相比于大型 Transformer 模型，它的上手门槛极低，部署资源消耗小，非常适合需要实时处理大规模文本流或在端侧设备运行的开发者。如果你不需要极致的学术精度，而是在意响应延迟和吞吐量，它是替代复杂 NLP 管线的理想选择。

核心亮点

  • 极速推理，适合大规模文本实时实体提取
  • 轻量化部署，显著降低显存和计算资源占用
  • 精准识别人名、地名、机构等核心英文实体
  • Apache-2.0 协议，支持灵活的商业化集成

使用方法

安装依赖
# 安装 Hugging Face transformers
pip install transformers torch
SDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("flair/ner-english-fast")
tokenizer = AutoTokenizer.from_pretrained("flair/ner-english-fast")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引 
pip install -U huggingface_hub

命令行下载

下载完整模型库

下载完整模型库 
huggingface-cli download flair/ner-english-fast

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例） 
huggingface-cli download flair/ner-english-fast config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

SDK 下载 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('flair/ner-english-fast')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://huggingface.co/flair/ner-english-fast

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/flair/ner-english-fast

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

安装 Transformers 
pip install -U transformers torch

模型加载和推理

模型加载和推理 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('flair/ner-english-fast')
tokenizer = AutoTokenizer.from_pretrained('flair/ner-english-fast')

模型下载

我们推荐使用命令行或者 ModelScope SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 ModelScope：

操作指引 
pip install modelscope

命令行下载

下载完整模型库

下载完整模型库 
modelscope download --model flair/ner-english-fast

下载单个文件到指定本地文件夹（以下载 README.md 到当前路径下 dir 目录为例）

下载单个文件到指定本地文件夹（以下载 README.md 到当前路径下 dir 目录为例） 
modelscope download --model flair/ner-english-fast README.md --local_dir ./dir

更多更丰富的命令行下载选项，可参见具体文档

SDK 下载

SDK 下载 
# 模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('flair/ner-english-fast')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://www.modelscope.cn/flair/ner-english-fast.git

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://www.modelscope.cn/flair/ner-english-fast.git

ModelScope 模型页直接下载模型文件；无需将模型文件放在本站服务器。

Notebook 快速开发

下载并安装 ModelScope library

下载并安装 ModelScope library 
pip install "modelscope[audio,cv,nlp,multi-modal,science]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html

模型加载和推理

模型加载和推理 
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

p = pipeline('text-generation', 'flair/ner-english-fast')

完整文档

来源: HuggingFace

---
tags:

  • flair

  • token-classification

  • sequence-tagger-model

language: en
datasets:
  • conll2003

widget:
  • text: "George Washington went to Washington"

---

English NER in Flair (fast model)

This is the fast 4-class NER model for English that ships with Flair.

F1-Score: 92,92 (corrected CoNLL-03)

Predicts 4 tags:

| tag | meaning |
|---------------------------------|-----------|
| PER | person name |
| LOC | location name |
| ORG | organization name |
| MISC | other name |

Based on Flair embeddings and LSTM-CRF.

---

Demo: How to use in Flair

Requires: Flair (pip install flair)

python
from flair.data import Sentence
from flair.models import SequenceTagger

load tagger

tagger = SequenceTagger.load("flair/ner-english-fast")

make example sentence

sentence = Sentence("George Washington went to Washington")

predict NER tags

tagger.predict(sentence)

print sentence

print(sentence)

print predicted NER spans

print('The following NER tags are found:')

iterate over entities and print

for entity in sentence.get_spans('ner'):
    print(entity)

This yields the following output:

code
Span [1,2]: "George Washington"   [− Labels: PER (0.9515)]
Span [5]: "Washington"   [− Labels: LOC (0.992)]

So, the entities "*George Washington*" (labeled as a person) and "*Washington*" (labeled as a location) are found in the sentence "*George Washington went to Washington*".

---

Training: Script to train this model

The following Flair script was used to train this model:

python
from flair.data import Corpus
from flair.datasets import CONLL_03
from flair.embeddings import WordEmbeddings, StackedEmbeddings, FlairEmbeddings

1. get the corpus

corpus: Corpus = CONLL_03()

2. what tag do we want to predict?

tag_type = 'ner'

3. make the tag dictionary from the corpus

tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

4. initialize each embedding we use

embedding_types = [

# GloVe embeddings
    WordEmbeddings('glove'),


# contextual string embeddings, forward
    FlairEmbeddings('news-forward-fast'),


# contextual string embeddings, backward
    FlairEmbeddings('news-backward-fast'),
]


embedding stack consists of Flair and GloVe embeddings

embeddings = StackedEmbeddings(embeddings=embedding_types)

5. initialize sequence tagger

from flair.models import SequenceTagger

tagger = SequenceTagger(hidden_size=256,
                        embeddings=embeddings,
                        tag_dictionary=tag_dictionary,
                        tag_type=tag_type)


6. initialize trainer

from flair.trainers import ModelTrainer

trainer = ModelTrainer(tagger, corpus)


7. run training

trainer.train('resources/taggers/ner-english',
              train_with_dev=True,
              max_epochs=150)

---

Cite

Please cite the following paper when using this model.

code
@inproceedings{akbik2018coling,
  title={Contextual String Embeddings for Sequence Labeling},
  author={Akbik, Alan and Blythe, Duncan and Vollgraf, Roland},
  booktitle = {{COLING} 2018, 27th International Conference on Computational Linguistics},
  pages     = {1638--1649},
  year      = {2018}
}

---

Issues?

The Flair issue tracker is available here.

查看官方来源