Name: bge m3
Author: BAAI

简介

BGE-M3 是由北京智源人工智能研究院（BAAI）推出的全能型向量模型，旨在解决传统 Embedding 模型在多语言支持和检索长度上的痛点。它最大的特色是“三合一”能力，同时支持稠密检索、稀疏检索和多向量检索，能够极大地提升 RAG（检索增强生成）系统的召回精度。对于开发者而言，它不仅支持 100 多种语言，还拥有 8192 tokens 的超长上下文窗口，上手门槛低，且与 LangChain、LlamaIndex 等主流 AI 框架无缝衔接，是构建企业级知识库的理想选择。

核心亮点

支持多语言混合检索，跨语言匹配能力强
8K 超长上下文，有效解决长文档截断问题
三路检索机制，显著提升 RAG 召回准确率
MIT 协议开源，易于私有化部署和商业集成

使用方法

安装依赖

# 安装 Hugging Face transformers
pip install transformers torch

SDK 使用

# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("BAAI/bge-m3")
tokenizer = AutoTokenizer.from_pretrained("BAAI/bge-m3")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引

pip install -U huggingface_hub

命令行下载

下载完整模型库

huggingface-cli download BAAI/bge-m3

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

huggingface-cli download BAAI/bge-m3 config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('BAAI/bge-m3')

Git 下载

请确保 lfs 已经被正确安装

Git 下载

git lfs install
git clone https://huggingface.co/BAAI/bge-m3

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/BAAI/bge-m3

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

pip install -U transformers torch

模型加载和推理

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('BAAI/bge-m3')
tokenizer = AutoTokenizer.from_pretrained('BAAI/bge-m3')

模型下载

我们推荐使用命令行或者 ModelScope SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 ModelScope：

操作指引

pip install modelscope

命令行下载

下载完整模型库

modelscope download --model BAAI/bge-m3

下载单个文件到指定本地文件夹（以下载 README.md 到当前路径下 dir 目录为例）

modelscope download --model BAAI/bge-m3 README.md --local_dir ./dir

更多更丰富的命令行下载选项，可参见具体文档

SDK 下载

# 模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('BAAI/bge-m3')

Git 下载

请确保 lfs 已经被正确安装

Git 下载

git lfs install
git clone https://www.modelscope.cn/BAAI/bge-m3.git

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS

GIT_LFS_SKIP_SMUDGE=1 git clone https://www.modelscope.cn/BAAI/bge-m3.git

ModelScope 模型页直接下载模型文件；无需将模型文件放在本站服务器。

Notebook 快速开发

下载并安装 ModelScope library

pip install "modelscope[audio,cv,nlp,multi-modal,science]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html

模型加载和推理

from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

p = pipeline('text-generation', 'BAAI/bge-m3')

完整文档

来源: HuggingFace

---
pipeline_tag: sentence-similarity
tags:

sentence-transformers

feature-extraction

sentence-similarity

license: mit
---

For more details please refer to our github repo: https://github.com/FlagOpen/FlagEmbedding

BGE-M3 (paper, code)

In this project, we introduce BGE-M3, which is distinguished for its versatility in Multi-Functionality, Multi-Linguality, and Multi-Granularity.

Multi-Functionality: It can simultaneously perform the three common retrieval functionalities of embedding model: dense retrieval, multi-vector retrieval, and sparse retrieval.

Multi-Linguality: It can support more than 100 working languages.

Multi-Granularity: It is able to process inputs of different granularities, spanning from short sentences to long documents of up to 8192 tokens.

Some suggestions for retrieval pipeline in RAG

We recommend to use the following pipeline: hybrid retrieval + re-ranking.

Hybrid retrieval leverages the strengths of various methods, offering higher accuracy and stronger generalization capabilities.

A classic example: using both embedding retrieval and the BM25 algorithm.
Now, you can try to use BGE-M3, which supports both embedding and sparse retrieval.
This allows you to obtain token weights (similar to the BM25) without any additional cost when generate dense embeddings.
To use hybrid retrieval, you can refer to Vespa and Milvus.

As cross-encoder models, re-ranker demonstrates higher accuracy than bi-encoder embedding model.

Utilizing the re-ranking model (e.g., bge-reranker, bge-reranker-v2) after retrieval can further filter the selected text.

News:

2024/7/1: We update the MIRACL evaluation results of BGE-M3. To reproduce the new results, you can refer to: bge-m3_miracl_2cr. We have also updated our paper on arXiv.

<details> <summary> Details </summary>

The previous test results were lower because we mistakenly removed the passages that have the same id as the query from the search results. After correcting this mistake, the overall performance of BGE-M3 on MIRACL is higher than the previous results, but the experimental conclusion remains unchanged. The other results are not affected by this mistake. To reproduce the previous lower results, you need to add the --remove-query parameter when using pyserini.search.faiss or pyserini.search.lucene to search the passages.

</details>

2024/3/20: Thanks Milvus team! Now you can use hybrid retrieval of bge-m3 in Milvus: pymilvus/examples

/hello_hybrid_sparse_dense.py.

2024/3/8: Thanks for the experimental results from @Yannael. In this benchmark, BGE-M3 achieves top performance in both English and other languages, surpassing models such as OpenAI.

2024/3/2: Release unified fine-tuning example and data

2024/2/6: We release the MLDR (a long document retrieval dataset covering 13 languages) and evaluation pipeline.

2024/2/1: Thanks for the excellent tool from Vespa. You can easily use multiple modes of BGE-M3 following this notebook

Specs

Model

| Model Name | Dimension | Sequence Length | Introduction |
|:----:|:---:|:---:|:---:|
| BAAI/bge-m3 | 1024 | 8192 | multilingual; unified fine-tuning (dense, sparse, and colbert) from bge-m3-unsupervised|
| BAAI/bge-m3-unsupervised | 1024 | 8192 | multilingual; contrastive learning from bge-m3-retromae |
| BAAI/bge-m3-retromae | -- | 8192 | multilingual; extend the max_length of xlm-roberta to 8192 and further pretrained via retromae|
| BAAI/bge-large-en-v1.5 | 1024 | 512 | English model |
| BAAI/bge-base-en-v1.5 | 768 | 512 | English model |
| BAAI/bge-small-en-v1.5 | 384 | 512 | English model |

Data

FAQ

1. Introduction for different retrieval methods

Dense retrieval: map the text into a single embedding, e.g., DPR, BGE-v1.5

Sparse retrieval (lexical matching): a vector of size equal to the vocabulary, with the majority of positions set to zero, calculating a weight only for tokens present in the text. e.g., BM25, unicoil, and splade

Multi-vector retrieval: use multiple vectors to represent a text, e.g., ColBERT.

2. How to use BGE-M3 in other projects?

For embedding retrieval, you can employ the BGE-M3 model using the same approach as BGE.
The only difference is that the BGE-M3 model no longer requires adding instructions to the queries.

For hybrid retrieval, you can use Vespa and Milvus.

3. How to fine-tune bge-M3 model?

You can follow the common in this example
to fine-tune the dense embedding.

If you want to fine-tune all embedding function of m3 (dense, sparse and colbert), you can refer to the unified_fine-tuning example

Usage

Install:

code

git clone https://github.com/FlagOpen/FlagEmbedding.git
cd FlagEmbedding
pip install -e .

or:

code

pip install -U FlagEmbedding

Generate Embedding for text

Dense Embedding

```python from FlagEmbedding import BGEM3FlagModel

model = BGEM3FlagModel('BAAI/bge-m3',
use_fp16=True) # Setting use_fp16 to True speeds up computation with a slight performance degradation

sentences_1 = ["What is BGE M3?", "Defination of BM25"]
sentences_2 = ["BGE M3 is an embedding model supporting dense retrieval, lexical matching and multi-vector interaction.",
"BM25 is a bag-of-words retrieval function that ranks a set of documents based on the query terms app

bge m3

简介

核心亮点

使用方法

Hugging Face 下载

命令行下载

SDK 下载

Git 下载

PyTorch / Transformers 使用

模型下载

命令行下载

SDK 下载

Git 下载

Notebook 快速开发

完整文档

BGE-M3 (paper, code)

News:

Specs

FAQ

Usage

Generate Embedding for text

扫码进交流群