Reducing LLM Hallucinations in Python Code Generation Using RAG-based Verification

Stop trusting the AI to remember your internal API signatures; it’s the fastest way to spend an hour debugging a TypeError that shouldn't exist. Even with Claude 3.5 Sonnet in Cursor, I noticed a recurring pattern where the LLM would hallucinate methods for my custom wrapper classes or use deprecated arguments from a library version it was trained on two years ago.

The fix isn't just "better prompting"—it's implementing a RAG-based verification loop that forces the AI to check its work against the actual source code before it presents the solution to you.

I've been prototyping a workflow where I use a local vector store (using ChromaDB) to index my project's documentation and the latest .py files. Instead of just asking the LLM to "write a function," I've shifted to a multi-step agentic flow.

Here is the logic I'm using in my custom scripts to wrap LLM calls:

import chromadb
from langchain_openai import OpenAIEmbeddings

# Quick setup for local context retrieval
client = chromadb.PersistentClient(path="./project_index")
collection = client.get_collection("api_specs")

def verify_and_generate(user_prompt):
    # Step 1: Initial generation
    initial_code = llm.invoke(f"Generate code for: {user_prompt}")
    
    # Step 2: Extract potential API calls from the generated code
    # I use a simple regex or a small LLM call to find calls to my internal modules
    calls = extract_internal_calls(initial_code)
    
    # Step 3: RAG Verification
    verification_context = ""
    for call in calls:
        docs = collection.query(query_texts=[call], n_results=1)
        verification_context += f"\nActual Signature: {docs['documents']}"
    
    # Step 4: Self-Correction
    final_code = llm.invoke(
        f"Original code: {initial_code}\n"
        f"Verified API signatures: {verification_context}\n"
        "Correct any hallucinations or signature mismatches."
    )
    return final_code

The productivity gain here is massive because it eliminates the "Trial and Error" loop. Instead of:
AI generates code → I run it → It crashes → I paste error back to AI → AI apologizes and fixes it,
the verification happens in the background.

A few config tips for those trying this:

Focus on the .pyi files. If you have type stubs, index those instead of the full .py implementation. The LLM doesn't need to see the logic of the function to know how to call it; it just needs the signature and the docstring. This reduces noise in the context window and prevents the model from getting distracted by the implementation details.

Use a "Strictness" prompt. In the second pass, I tell the model: "If the verified signature contradicts your initial code, you MUST prioritize the verified signature. Do not assume the documentation is wrong."

Watch out for token bloat. If you're indexing a massive repo, don't just dump everything into the prompt. I found that limiting the RAG retrieval to the top 2 most relevant snippets per function call keeps the latency low and prevents the model from losing the original intent of the prompt.

The biggest gotcha is the embedding quality. If you use a generic embedding model, it might struggle with camelCase or snake_case function names, thinking get_user_data and getUserData are different entities. I've found that adding the function name as a metadata tag in ChromaDB helps significantly with retrieval accuracy.