paraphrase multilingual MiniLM L12 v2 onnx Q

ProviderQdrant
Categorytext2text-generation
Licenseapache-2.0
Downloads499.0K
Stars0

Overview

The paraphrase-multilingual-MiniLM-L12-v2 (ONNX quantized) is a lightweight, high-efficiency sentence transformer designed for cross-lingual semantic similarity tasks. Unlike large generative models, this model focuses on mapping text from over 100 languages into a shared vector space, making it ideal for clustering, semantic search, and duplicate detection across different languages. The ONNX quantization significantly reduces the memory footprint and latency, allowing for high-throughput deployment on CPUs without requiring heavy GPU resources. For developers, this means an easy integration path for RAG pipelines or multilingual chatbots where low-latency embedding generation is critical.

Highlights

  • Optimized ONNX quantization for low-latency CPU inference
  • Supports semantic similarity across 100+ different languages
  • Ideal for RAG pipelines and multilingual vector search
  • Low memory overhead compared to full-precision transformers
  • Apache-2.0 license ensures flexible commercial integration

Usage

Install
# Install Hugging Face transformers
pip install transformers torch
SDK Usage
# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("Qdrant/paraphrase-multilingual-MiniLM-L12-v2-onnx-Q")
tokenizer = AutoTokenizer.from_pretrained("Qdrant/paraphrase-multilingual-MiniLM-L12-v2-onnx-Q")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance 
pip install -U huggingface_hub

CLI Download

Download the full repository

Download the full repository 
huggingface-cli download Qdrant/paraphrase-multilingual-MiniLM-L12-v2-onnx-Q

Download a single file to a local folder (e.g. config.json into ./dir)

Download a single file to a local folder (e.g. config.json into ./dir) 
huggingface-cli download Qdrant/paraphrase-multilingual-MiniLM-L12-v2-onnx-Q config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

SDK Download 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('Qdrant/paraphrase-multilingual-MiniLM-L12-v2-onnx-Q')

Git Download

Make sure git-lfs is installed first

Git Download 
git lfs install
git clone https://huggingface.co/Qdrant/paraphrase-multilingual-MiniLM-L12-v2-onnx-Q

To skip LFS large-file downloads, use:

Skip LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/Qdrant/paraphrase-multilingual-MiniLM-L12-v2-onnx-Q

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

Install Transformers 
pip install -U transformers torch

Load the model and run inference

Load the model and run inference 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('Qdrant/paraphrase-multilingual-MiniLM-L12-v2-onnx-Q')
tokenizer = AutoTokenizer.from_pretrained('Qdrant/paraphrase-multilingual-MiniLM-L12-v2-onnx-Q')

Full Documentation

来源: HuggingFace

---
license: apache-2.0
pipeline_tag: sentence-similarity
---

Quantized ONNX port of sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 for text classification and similarity searches.

Usage

Here's an example of performing inference using the model with FastEmbed.

py
from fastembed import TextEmbedding

documents = [
    "You should stay, study and sprint.",
    "History can only prepare us to be surprised yet again.",
]


model = TextEmbedding(model_name="sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2")
embeddings = list(model.embed(documents))


[


    array([1.96449570e-02, 1.60677675e-02, 4.10149433e-02...]),


    array([-1.56669170e-02, -1.66313536e-02, -6.84525725e-03...])


]
View official source
Join our Telegram