bert large portuguese cased

提供商neuralmind
分类fill-mask
许可证mit
下载量1.6M
星标0

简介

BERT Large Portuguese Cased 是由 Neuralmind 开发的针对葡萄牙语优化的预训练模型。它基于 BERT-Large 架构，在海量葡语语料上进行了深度训练，能够精准捕捉该语言的语法特性和上下文语义。对于需要处理葡语文本的开发者来说，它比通用多语言模型（如 mBERT）在语义理解和掩码预测任务上具有更高的准确率。该模型上手难度低，可直接通过 Hugging Face 等主流框架调用，非常适合作为下游任务（如情感分析、命名实体识别）的特征提取底座。

核心亮点

  • 深耕葡语语料，语义理解精度优于多语言模型
  • 支持 Cased 区分大小写，更精准处理专有名词
  • 典型的 Fill-Mask 结构，适配多种 NLP 下游任务
  • 采用 MIT 协议，商业化部署灵活且无压力

使用方法

安装依赖
# 安装 Hugging Face transformers
pip install transformers torch
SDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("neuralmind/bert-large-portuguese-cased")
tokenizer = AutoTokenizer.from_pretrained("neuralmind/bert-large-portuguese-cased")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引 
pip install -U huggingface_hub

命令行下载

下载完整模型库

下载完整模型库 
huggingface-cli download neuralmind/bert-large-portuguese-cased

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例） 
huggingface-cli download neuralmind/bert-large-portuguese-cased config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

SDK 下载 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('neuralmind/bert-large-portuguese-cased')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://huggingface.co/neuralmind/bert-large-portuguese-cased

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/neuralmind/bert-large-portuguese-cased

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

安装 Transformers 
pip install -U transformers torch

模型加载和推理

模型加载和推理 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('neuralmind/bert-large-portuguese-cased')
tokenizer = AutoTokenizer.from_pretrained('neuralmind/bert-large-portuguese-cased')

完整文档

来源: HuggingFace

---
language: pt
license: mit
tags:
- bert
- pytorch
datasets:
- brWaC
---

BERTimbau Large (aka "bert-large-portuguese-cased")

!Bert holding a berimbau

Introduction

BERTimbau Large is a pretrained BERT model for Brazilian Portuguese that achieves state-of-the-art performances on three downstream NLP tasks: Named Entity Recognition, Sentence Textual Similarity and Recognizing Textual Entailment. It is available in two sizes: Base and Large.

For further information or requests, please go to BERTimbau repository.

Available models

| Model | Arch. | #Layers | #Params |
| ---------------------------------------- | ---------- | ------- | ------- |
| neuralmind/bert-base-portuguese-cased | BERT-Base | 12 | 110M |
| neuralmind/bert-large-portuguese-cased | BERT-Large | 24 | 335M |

Usage

python
from transformers import AutoTokenizer  # Or BertTokenizer
from transformers import AutoModelForPreTraining  # Or BertForPreTraining for loading pretraining heads
from transformers import AutoModel  # or BertModel, for BERT without pretraining heads

model = AutoModelForPreTraining.from_pretrained('neuralmind/bert-large-portuguese-cased')
tokenizer = AutoTokenizer.from_pretrained('neuralmind/bert-large-portuguese-cased', do_lower_case=False)

Masked language modeling prediction example

python
from transformers import pipeline

pipe = pipeline('fill-mask', model=model, tokenizer=tokenizer)


pipe('Tinha uma [MASK] no meio do caminho.')
[{'score': 0.5054386258125305,

  'sequence': '[CLS] Tinha uma pedra no meio do caminho. [SEP]',

  'token': 5028,

  'token_str': 'pedra'},

 {'score': 0.05616172030568123,

  'sequence': '[CLS] Tinha uma curva no meio do caminho. [SEP]',

  'token': 9562,

  'token_str': 'curva'},

 {'score': 0.02348282001912594,

  'sequence': '[CLS] Tinha uma parada no meio do caminho. [SEP]',

  'token': 6655,

  'token_str': 'parada'},

 {'score': 0.01795753836631775,

  'sequence': '[CLS] Tinha uma mulher no meio do caminho. [SEP]',

  'token': 2606,

  'token_str': 'mulher'},

 {'score': 0.015246033668518066,

  'sequence': '[CLS] Tinha uma luz no meio do caminho. [SEP]',

  'token': 3377,

  'token_str': 'luz'}]

For BERT embeddings

python
import torch

model = AutoModel.from_pretrained('neuralmind/bert-large-portuguese-cased')
input_ids = tokenizer.encode('Tinha uma pedra no meio do caminho.', return_tensors='pt')


with torch.no_grad():
    outs = model(input_ids)
    encoded = outs[0][0, 1:-1]  # Ignore [CLS] and [SEP] special tokens


encoded.shape: (8, 1024)


tensor([[ 1.1872,  0.5606, -0.2264,  ...,  0.0117, -0.1618, -0.2286],


        [ 1.3562,  0.1026,  0.1732,  ..., -0.3855, -0.0832, -0.1052],


        [ 0.2988,  0.2528,  0.4431,  ...,  0.2684, -0.5584,  0.6524],


        ...,


        [ 0.3405, -0.0140, -0.0748,  ...,  0.6649, -0.8983,  0.5802],


        [ 0.1011,  0.8782,  0.1545,  ..., -0.1768, -0.8880, -0.1095],


        [ 0.7912,  0.9637, -0.3859,  ...,  0.2050, -0.1350,  0.0432]])

Citation

If you use our work, please cite:

bibtex
@inproceedings{souza2020bertimbau,
  author    = {F{\'a}bio Souza and
               Rodrigo Nogueira and
               Roberto Lotufo},
  title     = {{BERT}imbau: pretrained {BERT} models for {B}razilian {P}ortuguese},
  booktitle = {9th Brazilian Conference on Intelligent Systems, {BRACIS}, Rio Grande do Sul, Brazil, October 20-23 (to appear)},
  year      = {2020}
}
查看官方来源