paraphrase mpnet base v2

Providersentence-transformers
Categorysentence-similarity
Licenseapache-2.0
Downloads2.6K
Stars0

Overview

The paraphrase-mpnet-base-v2 is a high-performance sentence-transformer model optimized for mapping sentences and paragraphs to a dense vector space. Unlike general-purpose LLMs, this model is purpose-built for semantic similarity and clustering tasks, leveraging an MPNet architecture to balance the strengths of Masked Language Modeling (MLM) and Permuted Language Modeling (PLM). For developers, this means highly accurate embeddings that capture nuanced meaning rather than just keyword overlap. It is an ideal drop-in replacement for BERT-based encoders in RAG pipelines, semantic search engines, and duplicate detection systems. Integration is straightforward via the sentence-transformers library, offering a computationally efficient alternative to larger models while maintaining state-of-the-art retrieval performance on benchmark datasets.

Highlights

  • Optimized for high-accuracy semantic similarity and sentence embeddings
  • Ideal for RAG pipelines and semantic search indexing
  • Efficient MPNet architecture outperforms standard BERT encoders
  • Easy integration via sentence-transformers and Hugging Face
  • Permissive Apache-2.0 license for commercial production use

Usage

Install
# Install Hugging Face transformers
pip install transformers torch
SDK Usage
# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("sentence-transformers/paraphrase-mpnet-base-v2")
tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/paraphrase-mpnet-base-v2")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance 
pip install -U huggingface_hub

CLI Download

Download the full repository

Download the full repository 
huggingface-cli download sentence-transformers/paraphrase-mpnet-base-v2

Download a single file to a local folder (e.g. config.json into ./dir)

Download a single file to a local folder (e.g. config.json into ./dir) 
huggingface-cli download sentence-transformers/paraphrase-mpnet-base-v2 config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

SDK Download 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('sentence-transformers/paraphrase-mpnet-base-v2')

Git Download

Make sure git-lfs is installed first

Git Download 
git lfs install
git clone https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2

To skip LFS large-file downloads, use:

Skip LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

Install Transformers 
pip install -U transformers torch

Load the model and run inference

Load the model and run inference 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('sentence-transformers/paraphrase-mpnet-base-v2')
tokenizer = AutoTokenizer.from_pretrained('sentence-transformers/paraphrase-mpnet-base-v2')

Model Download

We recommend downloading the model via the ModelScope CLI or SDK.

Guidance：Before downloading, install ModelScope with:

Guidance 
pip install modelscope

CLI Download

Download the full repository

Download the full repository 
modelscope download --model sentence-transformers/paraphrase-mpnet-base-v2

Download a single file to a local folder (e.g. README.md into ./dir)

Download a single file to a local folder (e.g. README.md into ./dir) 
modelscope download --model sentence-transformers/paraphrase-mpnet-base-v2 README.md --local_dir ./dir

See the docs for more CLI options

SDK Download

SDK Download 
# 模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('sentence-transformers/paraphrase-mpnet-base-v2')

Git Download

Make sure git-lfs is installed first

Git Download 
git lfs install
git clone https://www.modelscope.cn/sentence-transformers/paraphrase-mpnet-base-v2.git

To skip LFS large-file downloads, use:

Skip LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://www.modelscope.cn/sentence-transformers/paraphrase-mpnet-base-v2.git

ModelScope 模型页直接下载模型文件；无需将模型文件放在本站服务器。

Notebook Quickstart

Install the ModelScope library

Install the ModelScope library 
pip install "modelscope[audio,cv,nlp,multi-modal,science]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html

Load the model and run inference

Load the model and run inference 
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

p = pipeline('text-generation', 'sentence-transformers/paraphrase-mpnet-base-v2')

Full Documentation

来源: HuggingFace

---
license: apache-2.0
library_name: sentence-transformers
tags:

  • sentence-transformers

  • feature-extraction

  • sentence-similarity

  • transformers

  • text-embeddings-inference

pipeline_tag: sentence-similarity
---

sentence-transformers/paraphrase-mpnet-base-v2

This is a sentence-transformers model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.

Usage (Sentence-Transformers)

Using this model becomes easy when you have sentence-transformers installed:

code
pip install -U sentence-transformers

Then you can use the model like this:

python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('sentence-transformers/paraphrase-mpnet-base-v2')
embeddings = model.encode(sentences)
print(embeddings)

Usage (HuggingFace Transformers)

Without sentence-transformers, you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.
python
from transformers import AutoTokenizer, AutoModel
import torch

Mean Pooling - Take attention mask into account for correct averaging

def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] # First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

Sentences we want sentence embeddings for

sentences = ['This is an example sentence', 'Each sentence is converted']

Load model from HuggingFace Hub

tokenizer = AutoTokenizer.from_pretrained('sentence-transformers/paraphrase-mpnet-base-v2')
model = AutoModel.from_pretrained('sentence-transformers/paraphrase-mpnet-base-v2')

Tokenize sentences

encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

Compute token embeddings

with torch.no_grad():
    model_output = model(**encoded_input)

Perform pooling. In this case, mean pooling.

sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)

Usage (Text Embeddings Inference (TEI))

Text Embeddings Inference (TEI) is a blazing fast inference solution for text embedding models.

  • CPU:
bash
docker run -p 8080:80 -v hf_cache:/data --pull always ghcr.io/huggingface/text-embeddings-inference:cpu-latest \
  --model-id sentence-transformers/paraphrase-mpnet-base-v2 \
  --pooling mean \
  --dtype float16
  • NVIDIA GPU:
bash
docker run --gpus all -p 8080:80 -v hf_cache:/data --pull always ghcr.io/huggingface/text-embeddings-inference:cuda-latest \
  --model-id sentence-transformers/paraphrase-mpnet-base-v2 \
  --pooling mean \
  --dtype float16

Send a request to /v1/embeddings to generate embeddings via the OpenAI Embeddings API:

bash
curl -s http://localhost:8080/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "sentence-transformers/paraphrase-mpnet-base-v2",
    "input": "This is an example sentence"
  }'

Or check the Text Embeddings Inference API specification instead.

Full Model Architecture

code
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)

Citing & Authors

This model was trained by sentence-transformers.

If you find this model helpful, feel free to cite our publication Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks:
``bibtex
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "http://arxiv.org/abs/1908.10084",
}
``

View official source
Join our Telegram