AI coding workflow

You can build a high-performance, local-first AI coding workflow by integrating Ollama for model hosting, VS Code with the Continue.dev extension, and a dedicated hardware setup containing at least 12GB of VRAM.

I spent all of last Tuesday trying to figure out why my Copilot subscription felt "laggy" compared to the local Llama 3 setup I ran on my workstation. The difference wasn't just preference; it was a measurable latency gap. My cloud-based autocomplete was hitting a 450ms delay per token, whereas my local Llama 3 8B model—running on an RTX 3060—responded in roughly 45ms. That speed difference changes how you write code. You stop thinking about the prompt and start thinking about the logic.

The Hardware Reality Check

Most people think they need a NASA supercomputer to run a decent AI coding workflow. You don't. You just need to understand the relationship between parameter count and Video RAM (VRAM).

If you try to run a 70B parameter model on an 8GB MacBook Air, you're going to have a bad time. It will swap to system memory, and your "real-time" coding assistant will suddenly feel like it's typing through molasses.

| Model Size | Minimum VRAM | Recommended VRAM | Latency (approx.) | Use Case |
| :--- | :--- | :--- | :--- | :--- |
| 3B (Phi-3) | 2.5 GB | 4 GB | < 20ms | Simple autocomplete |
| 8B (Llama 3) | 5 GB | 8-12 GB | 30-60ms | Refactoring & logic |
| 30B+ | 20 GB+ | 48 GB (Dual A6000) | 100ms+ | Complex architecture |

I realized early on that for 90% of my daily boilerplate and unit testing, a small, quantized 8B model is actually better than a massive, slow 70B model. Speed wins.

Setting up the local stack

To get this running, I use a specific stack that keeps my data off external servers. First, download Ollama. It’s the easiest way to manage local weights. Once that's running, you need a bridge to your editor.

I use the Continue.dev extension for VS Code. It's open-source and doesn't force you into a specific ecosystem. You configure it by pointing the models array in your config.json to your local Ollama endpoint.

{
  "models": [
    {
      "title": "Local Llama 3",
      "provider": "ollama",
      "model": "llama3:8b"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Starcoder2",
    "provider": "ollama",
    "model": "starcoder2:3b"
  }
}

The trick I learned—and it took a few failed attempts—is separating the "chat" model from the "autocomplete" model. Using a heavy model for autocomplete is a mistake. It creates a stutter in your typing. Use a tiny model like Starcoder2 for the ghost-text, and save the "smart" model for when you actually hit Cmd+L to ask a question about a function.

Why community knowledge beats documentation

Documentation tells you how a tool works. A community tells you why it breaks at 4:00 PM on a Friday.

When I hit a weird bug where my local LLM was hallucinating Python decorators that didn't exist, I didn't go to the official docs. I went to the PromptCube forums. People there aren't just posting "this is cool"; they are sharing specific quantization settings (like Q4_K_M vs Q8_0) that drastically change the accuracy of the output.

If you want to keep up with how different AI Models handle specific coding languages, you need to be where the people actually hitting the terminal hang out. PromptCube isn't a news site; it's a collection of practitioners. Joining means you stop guessing which model is best for Rust and start using the exact prompts others have already benchmarked.

Refining the prompt loop

Your AI coding workflow is only as good as your context management. If you just paste a 500-line file into a chat window, the model loses the plot.

I've started using a "Context-First" approach. Instead of saying "Fix this bug," I use the @file or @codebase features in Continue to provide the exact relevant snippets. It prevents the "lost in the middle" phenomenon where LLMs ignore instructions buried in massive blocks of text.

The wild part is how much better the results get when you stop treating the AI like a search engine and start treating it like a junior dev. If you give it vague instructions, you get vague code. If you give it specific constraints—like "use only the standard library" or "ensure O(n) complexity"—the output quality jumps significantly.

Avoiding the "Prompt Fatigue" trap

There is a tendency to over-engineer your setup. I've seen developers spend three weeks tweaking their Neovim config and local LLM weights only to end up writing less code than they did before.

Don't do that.

Start with a basic setup:
1. Ollama running in the background.
2. VS Code + Continue.
3. A decent GPU.

Once you feel the friction—whether it's the model being too slow or too "dumb"—that's when you iterate. If you find yourself struggling with how to structure your prompts to get better logic, that's the signal to dive into the AI Models discussions on PromptCube to see how others are fine-tuning their workflows.

The cost of "Free"

Local is great, but it isn't free. You are paying in electricity and hardware depreciation. I tracked my power draw with a smart plug during a heavy refactoring session. My desktop was pulling 350W. Over a month of heavy use, that adds up.

But when you compare that to a $20/month subscription plus the "mental cost" of your proprietary code living on a corporate server, the math usually favors the local setup for most professional developers.

The goal isn't to have the most expensive AI coding workflow. It's to have the one that disappears into your workflow so completely that you forget you're even using an AI.