splinter base
简介
核心亮点
- 专注问答场景，提供精准的知识提取与回答
- Apache-2.0 开源协议，商业化部署无压力
- 适合作为 RAG 架构中的生成端核心组件
- 轻量化设计，降低企业私有化部署门槛
使用方法
# 安装 Hugging Face transformers
pip install transformers torch# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("tau/splinter-base")
tokenizer = AutoTokenizer.from_pretrained("tau/splinter-base")Hugging Face 下载
我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。
操作指引：在下载前，请先通过如下命令安装 huggingface_hub：
pip install -U huggingface_hub命令行下载
下载完整模型库
huggingface-cli download tau/splinter-base下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）
huggingface-cli download tau/splinter-base config.json --local-dir ./dirSDK 下载
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('tau/splinter-base')Git 下载
请确保 lfs 已经被正确安装
git lfs install
git clone https://huggingface.co/tau/splinter-base如果您希望跳过 lfs 大文件下载，可以使用如下命令
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/tau/splinter-base模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。
PyTorch / Transformers 使用
安装 Transformers
pip install -U transformers torch模型加载和推理
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained('tau/splinter-base')
tokenizer = AutoTokenizer.from_pretrained('tau/splinter-base')完整文档
---
language: en
tags:
- splinter
- SplinterModel
license: apache-2.0
---
Splinter base model
Splinter-base is the pretrained model discussed in the paper Few-Shot Question Answering by Pretraining Span Selection (at ACL 2021). Its original repository can be found here. The model is case-sensitive.
Note: This model doesn't contain the pretrained weights for the QASS layer (see paper for details), and therefore the QASS layer is randomly initialized upon loading it. For the model with those weights, see tau/splinter-base-qass.
Model description
Splinter is a model that is pretrained in a self-supervised fashion for few-shot question answering. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from those texts.
More precisely, it was pretrained with the Recurring Span Selection (RSS) objective, which emulates the span selection process involved in extractive question answering. Given a text, clusters of recurring spans (n-grams that appear more than once in the text) are first identified. For each such cluster, all of its instances but one are replaced with a special [QUESTION] token, and the model should select the correct (i.e., unmasked) span for each masked one. The model also defines the Question-Aware Span selection (QASS) layer, which selects spans conditioned on a specific question (in order to perform multiple predictions).
Intended uses & limitations
The prime use for this model is few-shot extractive QA.
Pretraining
The model was pretrained on a v3-8 TPU for 2.4M steps. The training data is based on Wikipedia and BookCorpus. See the paper for more details.
BibTeX entry and citation info
@inproceedings{ram-etal-2021-shot,
title = "Few-Shot Question Answering by Pretraining Span Selection",
author = "Ram, Ori and
Kirstain, Yuval and
Berant, Jonathan and
Globerson, Amir and
Levy, Omer",
booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)",
month = aug,
year = "2021",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2021.acl-long.239",
doi = "10.18653/v1/2021.acl-long.239",
pages = "3066--3079",
}