Optimizing Ollama Local Deployment for Faster Python Code Generation and Autocomplete

Running LLMs locally via Ollama is great for privacy, but the latency on Python autocomplete can be a killer if you're using a model that's too heavy for your VRAM. I've spent the last month tweaking my setup to get that "instant" feel I get with Claude or Copilot, and the secret isn't just about the hardware—it's about model selection and specific Ollama environment tweaks.

The biggest mistake is trying to use a general-purpose 7B or 13B model for real-time ghost text. For Python autocomplete, I switched to deepseek-coder:1.3b. It's tiny, fits entirely in the GPU cache, and the TTFT (Time to First Token) is nearly zero. For actual logic generation or refactoring, I swap to codestral or deepseek-coder:33b (via 4-bit quantization), but I never use the big models for the autocomplete trigger.

If you're seeing sluggish responses even with small models, check your OLLAMA_NUM_PARALLEL and OLLAMA_MAX_LOADED_MODELS settings. By default, Ollama might unload your autocomplete model to make room for a larger one you used for a chat query, leading to a 2-3 second "wake up" delay.

On macOS or Linux, you can set these environment variables to keep your coding models resident in memory:

# Keep the autocomplete model loaded indefinitely
export OLLAMA_KEEP_ALIVE=24h
# Allow multiple models to stay in VRAM if you have the space
export OLLAMA_MAX_LOADED_MODELS=3

To integrate this into VS Code, I stopped using the generic extensions and moved to Continue.dev. The config is where you actually gain the productivity. You want to separate your tabAutocompleteModel from your chatModel.

Here is the exact snippet from my config.json that optimizes the Python loop:

{
  "models": [
    {
      "title": "Codestral",
      "model": "codestral",
      "provider": "ollama"
    }
  ],
  "tabAutocompleteModel": {
    "title": "DeepSeek 1.3B",
    "model": "deepseek-coder:1.3b",
    "provider": "ollama"
  },
  "embeddingsProvider": {
    "provider": "ollama",
    "model": "nomic-embed-text"
  }
}

One major "gotcha" I hit: Ollama's default context window can sometimes cause a performance dip as the file grows. If you notice the autocomplete slowing down in 500+ line Python files, you need to create a custom Modelfile to cap the context window for the autocomplete model.

Run this in your terminal:

# Create a streamlined version of the coder model
ollama show deepseek-coder:1.3b --modelfile > coder.modelfile
# Edit coder.modelfile and add:
# PARAMETER num_ctx 4096
ollama create ds-coder-fast -f coder.modelfile

Then point your autocomplete config to ds-coder-fast. Reducing the context from the default to 4k for autocomplete significantly reduces the KV cache overhead and keeps the token generation snappy.

My current performance gains:

• Cold start to first token: Reduced from 2.5s to ~0.2s.
• RAM Overhead: 1.3B model uses roughly 1.2GB VRAM, leaving plenty of room for the IDE and browser.
• Accuracy: For boilerplate and standard library calls, the 1.3B model is surprisingly precise; I only trigger the 33B model via @codebase when I'm architecting a new module.