Aurora: A Go AI Gateway for Routing, Caching, and Cost Control

If you've ever wired up OpenAI and Anthropic inside the same service, you already know the part that eats your day: API keys, provider SDKs, retry logic, cost tracking. That's not application logic, it's plumbing. Aurora treats it as an infra layer, and after running it through a few real-world test scenarios, I think it actually delivers on that promise.

The build is a single ~15MB Go binary with zero runtime dependencies. You point it at your providers, give it a config, and it starts proxying requests to OpenAI, Anthropic, Gemini, and others. The neat part: it converts between wire formats on the fly. Your backend speaks the OpenAI SDK, Aurora translates to Anthropic's format automatically, so you don't maintain conversion code in every microservice. That alone simplifies a lot of deployments.

Here's what it handles out of the box:

• Cost tracking per customer or team, so you can see who's actually burning tokens
• Model switching across providers without touching application code
• Debugging request and response flows through a single gateway point
• Exact and semantic caching to cut repeated calls and stay under budget

What separates it from other gateways I've tried:

• Auto-discovery of provider models — no manual model catalog maintenance
• Provider pools with automatic failover, so a 429 or outage doesn't kill your feature
• Built-in guardrails, audit logging with export, analytics, and an admin dashboard
• Model aliases that let you map "my-small" to whatever currently makes sense
• Regional deployment to shave latency by routing closer to the user
• Admin endpoints for full control, including creating managed API keys during user onboarding

The OSS version is Apache 2.0 and includes everything listed above. Enterprise adds SSO, RBAC, and tenant isolation for larger orgs that need tighter access control.

Installation is straightforward:

npm install -g iaurora

I'm not going to pretend it replaces every piece of your integration strategy — if you need extremely niche provider features, you may still want direct SDK calls. But for teams juggling multiple LLM providers and per-customer cost visibility, this slot in the stack is exactly where the complexity should live. The fact that it's a single Go binary and not a Python service is a bonus for anyone who's tired of babysitting dependency trees.

Give it a spin on your next side project, especially if you're doing a hands-on evaluation of LLM gateways. The default setup takes minutes, and the failover behavior is noticeably smoother than writing your own retry loop from scratch.