Optimizing Qwen2.5-Coder for Local Python Automation with Ollama and LangGraph

Qwen2.5-Coder is arguably the best open-weight model for Python right now, especially when you need it to handle complex logic without sending your local file system data to a cloud API. I've been pairing it with Ollama and LangGraph to build a self-correcting automation agent that manages my local logs and database migrations, and the "loop" performance is surprisingly tight.

The biggest hurdle with local LLMs in agentic workflows is the "hallucination loop"—where the model writes a bug, the executor throws an error, and the model repeats the same wrong fix. To kill this, I stopped using generic prompts and started feeding the model its own execution failures as a strict constraint.

Here is the setup I'm using to keep the agent on track:

The Ollama Config
I noticed the default 4k context window in Ollama can cause the model to lose track of the codebase in longer LangGraph cycles. I bumped the num_ctx to 32k in a custom Modelfile to ensure it can see the full stack trace and the previous three attempts at a fix.

# Create a custom model file
nano Modelfile

FROM qwen2.5-coder:7b
PARAMETER num_ctx 32768
PARAMETER temperature 0.2
SYSTEM "You are a Python automation expert. Output ONLY valid Python code within blocks. Do not explain your reasoning unless asked."

Then I bake it: ollama create qwen-automation -f Modelfile.

The LangGraph State Logic
The secret to making Qwen2.5-Coder actually "work" locally is a tight feedback loop. I structured my graph with a coder node and an executor node. If the executor catches a subprocess.CalledProcessError, it feeds the stderr back to the coder with a specific prefix: CRITICAL_ERROR: [error message]. This triggers the model's debugging mode much better than a polite "please fix this."

from langgraph.graph import StateGraph, END
from typing import TypedDict

class AgentState(TypedDict):
    code: str
    error: str
    iterations: int

def coder_node(state: AgentState):
    # Using LangChain's Ollama integration
    prompt = f"Fix this code: {state['code']}\nError: {state['error']}"
    response = llm.invoke(prompt)
    return {"code": response.content, "iterations": state['iterations'] + 1}

def executor_node(state: AgentState):
    try:
        exec(state['code'])
        return {"error": None}
    except Exception as e:
        return {"error": str(e)}

# Graph construction
workflow = StateGraph(AgentState)
workflow.add_node("coder", coder_node)
workflow.add_node("executor", executor_node)
workflow.set_entry_point("coder")
workflow.add_edge("coder", "executor")
workflow.add_conditional_edges("executor", lambda x: "coder" if x["error"] else END)

Productivity Gains & Gotchas
Performance: On an RTX 3090, the 7B model responds almost instantly. The latency is low enough that I can actually iterate on a script in real-time without the "waiting for API" lag.
The "Infinite Loop" Trap: Local models can get stuck. I always hard-code a max_iterations=5 limit in the state. If it hits 5, I have the system dump the current state to a .txt file so I can manually intervene.
Precision: Qwen2.5-Coder is great at syntax, but occasionally misses the specific version of a local library. I found that adding a "Environment Check" step at the start of the graph—where the agent runs pip freeze and reads the output—drastically reduces import errors.

If you're running this on a Mac M2/M3, stick to the 7B version. The 32B is smarter, but the token-per-second drop makes the LangGraph loop feel sluggish, which kills the flow of automation.