LiteLLM Gateway Hijacking: When Your AI Proxy Turns Rogue

Alright, let's talk about a genuinely worrying attack vector that's been flying under the radar. LiteLLM isn't just a convenience wrapper anymore — it's become the backbone of AI infrastructure for hundreds of teams, and that makes it a juicy target.

Here's the deal: LiteLLM sits between your application and the LLM providers. It handles routing, manages API keys, and normalizes responses across different vendors. But because it has that privileged position and stores those precious backend credentials in memory, it becomes the perfect man-in-the-middle candidate.

The Core Problem

What's particularly nasty about this isn't just data exfiltration — it's response manipulation. Since LiteLLM processes every request and response, an attacker who gains access can:

• Intercept and read all LLM conversations
• Steal API keys for every provider the gateway touches
• Modify responses before they reach the end user
• Inject malicious tool calls that the LLM thinks are legitimate

The Attack Surface

The most common exploitation path involves compromising the LiteLLM instance itself — either through container escape, misconfigured permissions, or supply chain attacks targeting the deployment pipeline. Once inside, the attacker has direct access to the Python runtime where LiteLLM executes.

From there, the techniques are remarkably straightforward:

1. Memory scraping — API keys are stored in process memory, often in plaintext or easily reversible formats
2. Traffic interception — Monkey-patching the HTTP client or intercepting at the network level
3. Response tampering — Hooking into the response processing pipeline to rewrite outputs
4. Tool call injection — Inserting malicious function calls into the tool execution chain

Why This Matters More Than Ever

We're seeing organizations route everything through LiteLLM — embeddings, chat completions, tool calls, you name it. The gateway has become the chokepoint for AI policy enforcement, rate limiting, and cost management. That concentration of power means compromising it gives you influence over an entire organization's AI interactions.

Defenders should be monitoring for:

• Unusual outbound network connections from LiteLLM containers
• Unexpected memory access patterns or process injection attempts
• Modifications to the LiteLLM Python package or its dependencies
• Anomalous tool call patterns or response structures

The really concerning part? Most teams treat their AI gateway as a solved problem once it's running. Nobody's actively threat-modeling the fact that their LLM proxy is essentially a credential vault with internet access.

This isn't theoretical anymore. The techniques exist, the targets are ripe, and the impact could be catastrophic for organizations that don't start treating their AI infrastructure with the same scrutiny they give their databases.