Optimizing RAG Retrieval Performance by Tuning Milvus Index Parameters

Most people treat Milvus indexes as a "set and forget" configuration, but if your RAG pipeline is returning irrelevant chunks or lagging under load, the default index settings are usually the culprit. I spent the last few days debugging a retrieval latency spike in a project with about 10 million vectors, and it turned out that my HNSW parameters were completely mismatched for the data distribution.

The core trade-off in Milvus (especially with HNSW) is between M (max degree of a node) and efConstruction (search scope during index building). If you leave these at defaults, you're often sacrificing recall for speed without knowing it.

For high-precision RAG, I’ve found that bumping M to 32 or 64 significantly improves the "reachability" of the graph, which helps when your embeddings are clustered tightly. But the real magic happens with ef during search.

Here is how I currently configure my index for a balance of speed and accuracy:

from pymilvus import Collection

collection = Collection("knowledge_base")

# Optimizing for high recall in a production RAG environment
index_params = {
    "index_type": "HNSW",
    "metric_type": "L2", 
    "params": {
        "M": 32,                # Increased from 16 for better connectivity
        "efConstruction": 256   # Higher value = better index quality, slower build
    }
}

collection.create_index(
    field_name="vector", 
    index_params=index_params
)

One huge "gotcha" that isn't obvious from the docs: the ef parameter during search is separate from efConstruction. If your RAG system is missing the correct context despite the document being in the DB, don't jump to changing your embedding model immediately. Try increasing the search ef first.

# This is where the actual retrieval precision is tuned
search_params = {
    "metric_type": "L2", 
    "params": {
        "ef": 64 # Increase this to improve recall at the cost of latency
    }
}

results = collection.search(
    data=[query_vector], 
    anns_field="vector", 
    param=search_params, 
    limit=5, 
    expr=None
)

My current productivity workflow for tuning this involves a "Recall Sweep." I take a golden dataset of 100 query-document pairs, run the search with ef ranging from 10 to 200, and plot the latency vs. recall. In my last project, I found that moving ef from 16 to 64 improved my hit rate by 12% while only adding 4ms to the response time.

A few concrete tips for different workloads:

High-throughput / Low-latency: Stick to IVF_FLAT if your dataset is small enough to fit in memory. It avoids the graph overhead of HNSW.

Massive Datasets: If you're hitting memory ceilings, use IVF_PQ. Just be warned that Product Quantization (PQ) compresses vectors, so you'll see a dip in precision. You'll need to tune nlist carefully—usually, $\sqrt{N}$ (where $N$ is the number of vectors) is the starting point, but I usually go slightly higher to keep search buckets small.

Memory Optimization: Always check your index_type against your available RAM. HNSW is a memory hog because it stores the graph structure alongside the vectors. If your pods are OOMing, that's the first place to look.