Ollama Deployment: A Complete Guide

Local LLM execution is often marketed as a "seamless" experience, but the reality depends entirely on your VRAM and how you handle environment variables. If you're tired of cloud latency or privacy concerns with APIs, Ollama is currently the most streamlined way to handle local weights, provided you know how to actually configure the backend.

Installation and Setup

The installation process varies by OS, but the core binary remains the same. For those on Linux, the systemd integration is the only way to ensure the service persists across reboots.

Linux Deployment

Run the official installation script:

curl -fsSL https://ollama.com/install.sh | sh

To verify the installation and check the service status:

ollama -v
sudo systemctl status ollama
sudo systemctl enable ollama
sudo systemctl restart ollama

If you are running an AMD setup, you'll need the ROCm version specifically:

curl -L https://ollama.com/download/ollama-linux-amd64-rocm.tgz -o ollama-linux-amd64-rocm.tgz
sudo tar -C /usr/ -xzf ollama-linux-amd64-rocm.tgz

macOS and Windows

For macOS, drag the Ollama.app to your Applications folder. For Windows, use the .exe installer. Once installed, you can interact with the CLI via Terminal or PowerShell:

ollama -v
ollama list
ollama serve

Model Management and Benchmarking

The pull command is where most users run into disk space issues. Be cautious with larger weights; for example, Llama 3.3 requires ~40GB of storage.

Pulling and listing models:

ollama pull mistral
ollama pull deepseek-r1:1.5b
ollama pull llama3.3
ollama list

Running models for comparison:
I typically run the same prompt across different model sizes to check the trade-off between token speed (tokens per second) and reasoning quality.

ollama run qwen2.5:1.5b

For reasoning-heavy tasks, DeepSeek R1 is the current benchmark. It outputs its internal chain-of-thought within <think> blocks, which is invaluable for debugging why a model reached a specific (and potentially wrong) conclusion.

ollama run deepseek-r1:1.5b

Essential management commands:

• Inspect params: ollama show llama3.3 (Check context length and license here)
• Kill process: ollama stop deepseek-r1:1.5b
• Cleanup: ollama rm mistral

Advanced Configuration via Environment Variables

Default settings are rarely optimal for power users. If you want to expose your local LLM to other machines on your network or optimize VRAM usage, you must modify the environment variables.

Key Variables for Tuning:

• OLLAMA_HOST: Set to 0.0.0.0:11434 to allow remote network access.
• OLLAMA_MODELS: Change this if your root drive is too small and you need to store weights on a secondary SSD.
• OLLAMA_KEEP_ALIVE: Controls how long a model stays resident in VRAM.
• OLLAMA_FLASH_ATTENTION: Enable this for experimental performance gains on supported GPUs.

Linux Configuration Example

You cannot just export these in the shell; you must edit the systemd unit file for them to persist:

sudo vim /etc/systemd/system/ollama.service

Add the following under the [Service] section:

[Service]
Environment="OLLAMA_DEBUG=1"
Environment="OLLAMA_HOST=0.0.0.0:11434"

Apply the changes:

sudo systemctl daemon-reload
sudo systemctl restart ollama

macOS and Windows Setup

On macOS, use launchctl setenv OLLAMA_HOST "0.0.0.0". On Windows, navigate to Edit the System Environment Variables and add these as new User variables.

This setup transforms Ollama from a simple CLI tool into a proper local LLM agent backend that can be integrated into larger AI workflows.