Geopolitics and the race for AI inf

Hardware bottlenecks are the real "hidden boss" of the AI era, and as someone spending 10+ hours a day in Cursor and Claude Code, I’ve noticed that the race for compute infrastructure directly dictates which models actually feel "snappy" and which ones hallucinate because they're being served on suboptimal quantized shards.

When we talk about "AI infra," most people think of data centers, but for us developers, it manifests as token latency and context window stability. I’ve been experimenting with local LLM orchestration using Ollama and vLLM to offload some of my boilerplate tasks from the cloud, mainly because the geopolitical scramble for H100s has made API pricing volatile and rate limits unpredictable.

If you're trying to build a resilient local dev stack to avoid being totally dependent on a single provider's cloud region, here is how I’ve configured my local "fallback" environment. I use a dedicated Linux box with a couple of 3090s (since they are the best bang-for-buck for local VRAM) to run DeepSeek-Coder or Llama 3.

To get an AI-powered IDE experience without hitting the "Rate Limit Exceeded" wall during peak US hours, I route my local models through a LiteLLM proxy. This makes the local model look like an OpenAI endpoint, which Cursor can then hook into:

# Install LiteLLM to proxy local Ollama to OpenAI format
pip install 'litellm[all]'

# Start the proxy pointing to your local Ollama instance
litellm --model ollama/deepseek-coder:33b

In Cursor's settings, I just point the Base URL to http://localhost:4000. This setup is a lifesaver when regional outages happen or when you're working with sensitive code that shouldn't leave your network.

One major "gotcha" I've encountered is the discrepancy between FP16 and 4-bit quantized models. When you're using these "democratized" versions of models (because the full-weight versions are locked behind massive compute clusters), the logic for complex refactoring often breaks. I've found that for Python, the 4-bit quants are fine, but for TypeScript/React, the loss in precision leads to weird syntax errors that the AI then spends five turns trying to "fix" without success.

To mitigate this, I’ve started using a "hybrid prompt" strategy. I use the high-end cloud models for the architectural design and the local infra for the repetitive implementation.

My current productivity workflow:

• Architecture/Complex Logic: Claude 3.5 Sonnet via Cursor (Cloud).
• Boilerplate/Unit Tests: DeepSeek-Coder via local vLLM (Local Infra).
• Documentation/Comments: Llama 3 via Ollama (Local Infra).

The reality is that the "AI race" isn't just about who has the best algorithm, but who has the most stable power grid and the most silicon. By diversifying where your "intelligence" comes from—mixing cloud APIs with local self-hosted infra—you stop being a victim of whoever happens to be winning the chip war this month.

If you're setting up local infra, don't overlook the importance of NVMe speeds. I noticed a significant jump in "Time to First Token" (TTFT) just by moving my model weights from a SATA SSD to a Gen4 NVMe. It's a small hardware tweak, but it makes the difference between a tool that feels like an extension of your brain and one that feels like a slow chat bot.