Optimizing Hybrid Search Performance Using BGE Embeddings and Milvus for RAG

Cursor's Composer mode has completely changed how I iterate on RAG pipelines; instead of manually writing boilerplate for vector database connections, I just feed it the Milvus API docs and let it scaffold the hybrid search logic. The real struggle isn't the plumbing, but the "tuning" phase where you realize your semantic search is hallucinating and your keyword search is too rigid.

I've been benchmarking BGE-M3 embeddings because they handle multi-linguality and long sequences way better than the standard OpenAI embeddings. When pairing BGE with Milvus, the magic happens in the hybrid search configuration—specifically how you handle the Weighted Ranker. If you just let the system default, you'll find that dense vectors often drown out the precise keyword matches (BM25).

Here is the exact pattern I use to balance the two. I found that a 0.7 dense / 0.3 sparse split is the sweet spot for technical documentation, where specific function names must be matched exactly, but the intent is semantic.

from pymilvus import MilvusClient, AnnSearchRequest, RRFRanker

client = MilvusClient("http://localhost:19530")

# BGE-M3 generates both dense and sparse vectors
# Ensure your collection schema has both fields
res = client.hybrid_search(
    collection_name="tech_docs",
    reqs=[
        AnnSearchRequest(
            data=[dense_vector], 
            anns_field="dense_vector", 
            param={"metric_type": "L2", "params": {"nprobe": 10}}, 
            limit=10
        ),
        AnnSearchRequest(
            data=[sparse_vector], 
            anns_field="sparse_vector", 
            param={"metric_type": "BGE_M3_SPARSE"}, 
            limit=10
        )
    ],
    ranker=RRFRanker(w=[0.7, 0.3]), # Weighting semantic higher but keeping keyword influence
    limit=5
)

One massive gotcha with BGE-M3 is the token limit. While it claims to support 8k tokens, the retrieval quality drops off a cliff after 512 tokens. I spent three days wondering why my long-form PDF chunks were returning garbage until I realized I needed to implement a "sliding window" chunking strategy.

To automate this, I used Claude 3.5 Sonnet via Cursor to write a custom recursive character splitter that preserves semantic boundaries (like not splitting in the middle of a code block).

My current optimization checklist for this stack:

• Index Selection: Use HNSW for the dense field. It's the only way to keep latency under 50ms when the dataset hits 1M+ vectors.
• Normalization: Always normalize BGE embeddings before ingestion if you're using Cosine similarity, otherwise, the distance calculations get wonky.
• Sparse Vector Storage: Use the SPARSE_FLOAT_VECTOR type in Milvus to avoid wasting memory on zeros.

The productivity gain here is mostly in the "loop." I use the @Codebase feature in Cursor to index my entire Milvus implementation, then I prompt: "Compare the recall rate of the current RRFRanker weights against a pure dense search for the last 10 queries in logs.txt". It writes the evaluation script in seconds, allowing me to tune weights based on actual data rather than guessing.

If you're seeing "noisy" results, stop tweaking the embeddings and look at your reranker. Adding a BGE-Reranker step after the hybrid search is non-negotiable for production-grade RAG. It takes the top 20 candidates from Milvus and re-scores them, which usually kills 80% of the hallucinations.