Limited AI tooling forced our team to master prompting

Last quarter leadership handed down a mandate: integrate LLMs into every product squad. The first thing our platform team did was spin up access to OpenAI, Anthropic, Cohere, Mistral, a couple of fine-tuned Llamas, and three internal checkpoints — "so everyone can experiment." Two months later, adoption was flat. Engineers spent more time debating which model to call than actually shipping features.

Sound familiar?

I saw the same paralysis at a Cursor meetup last week. A non-technical attendee asked how anyone navigates the tool explosion without drowning. He assumed more options meant better outcomes. I didn't have a clean answer until I pulled UNIX Network Programming off a library shelf that weekend.

Stevens documents the sockets API: socket(), bind(), listen(), accept(), connect(), read(), write(), close(). That's it. Eight calls. Built in 1983 on machines with kilobytes of RAM, MHz clocks, and flaky 56 kbps links. No room for sprawling interfaces or generic contracts. Every function had to earn its keep.

Yet that constraint produced TCP/IP, the internet backbone, and a generation of engineers who understood the stack because they couldn't hide behind abstraction layers.

We did the same thing internally. Froze the model roster at three: GPT-4o for reasoning-heavy tasks, Claude 3.5 Sonnet for code, and a small distilled Llama for low-latency classification. Killed the rest. Told the squads: "Here's your vocabulary. Build."

Two things happened fast.

First, prompt engineering became a shared craft instead of a solo experiment. Teams started swapping few-shot templates, chain-of-thought patterns, and eval harnesses because they were all speaking the same dialect. Our internal prompt library grew from zero to forty verified snippets in three weeks.

Second, latency and cost dropped 40% because nobody was A/B testing five models per feature. We added a lightweight router that picks the right model based on task tags — reasoning, coding, classify — and logs every call for offline eval.

router:
  reasoning: gpt-4o
  coding: claude-3.5-sonnet
  classify: llama-3.1-8b-instruct

The constraint didn't limit creativity; it forced us to go deep on the tools we kept. Engineers now debug token budgets the way kernel hackers once debugged mbuf chains.

If your AI rollout feels scattered, try subtracting. Pick a tiny, opinionated toolset. Document the hell out of it. Watch what gets built when the only way forward is through the door in front of you.