Optimizing Ollama Performance for Local Llama 3 Deployment on Mac M3

Running Llama 3 on an M3 Mac via Ollama is generally smooth, but if you're noticing latency spikes or high memory pressure when multitasking, you're likely hitting the limits of how Ollama manages the Unified Memory Architecture (UMA). The key to squeezing more performance out of the M3 isn't just about the model size, but how you handle the context window and the GPU offloading.

The biggest performance killer on macOS is memory swapping. If you try to run a 70B quantized model on a 24GB or 36GB RAM machine, macOS will swap to the SSD, and your tokens-per-second (t/s) will plummet. For the 8B model, the default settings are fine, but for larger variants, you need to be surgical with your Modelfile.

I've found that creating a custom Modelfile allows you to tweak the num_ctx (context window) to prevent the model from eating all available VRAM. By default, Ollama might allocate more than you need, which limits the space available for KV cache during long conversations.

Here is how I optimize my local Llama 3 instance:

1. Create a file named Modelfile:

FROM llama3:8b
# Reduce context window to 4096 to save memory and increase speed
PARAMETER num_ctx 4096
# Adjust temperature for more consistent coding outputs
PARAMETER temperature 0.2
# Set the system prompt to force concise answers
SYSTEM "You are a senior engineer. Give concise, technical answers without fluff."

2. Build and run the optimized version:

ollama create llama3-fast -f Modelfile
ollama run llama3-fast

If you're experiencing "stuttering" in the output, check your Activity Monitor. If ollama_model_runner is pushing your memory into the "yellow" zone, you're over-provisioning. A pro tip for M3 users: keep your browser tabs (especially Chrome) to a minimum when running heavy local LLMs. The M3's unified memory is shared; if your browser is eating 8GB, that's 8GB less for the GPU to handle the model weights.

Another gotcha is the "keep_alive" setting. By default, Ollama keeps the model in memory for 5 minutes. If you're switching between different models (e.g., Llama 3 for logic and Mistral for summaries), this can lead to memory fragmentation. You can force the model to unload immediately after a request by passing an environment variable or using the API:

# Set keep_alive to 0 to unload immediately after response
curl http://localhost:11434/api/generate -d '{
  "model": "llama3",
  "prompt": "Why is unified memory better for LLMs?",
  "keep_alive": 0
}'

For those using the M3 Max with high memory bandwidth, you can actually push the context window higher, but I've noticed that num_gpu is usually handled automatically by Ollama on Mac. If you feel the CPU is doing too much work, ensure you aren't running other GPU-heavy apps like Lightroom or Blender in the background.

Concrete productivity gains I've seen:
Token Speed: Dropping num_ctx from 8k to 4k on the 8B model increased my perceived response start time (Time to First Token) by about 20%.
Stability: Using a custom Modelfile with a lower temperature (0.2) stopped the model from "rambling," which effectively reduced the number of tokens generated per response, making the overall interaction feel snappier.
Resource Management: Setting keep_alive: 0 stopped my Mac from lagging during heavy IDE usage when I wasn't actively querying the LLM.