Revoking LLM database access is where the real work starts

We learned this the hard way last quarter. Our analytics pipeline needed a quick natural-language layer for the product team. Spun up a service account, granted SELECT on the warehouse, pointed the LLM at it. Worked beautifully — until the vendor announced a pricing change and we needed to rotate keys across three environments.

The model itself doesn't hold credentials. But the orchestration layer does. And that layer talks to secrets managers, IAM policies, Terraform state, and a half-dozen internal tools that all assume the service account is permanent.

What actually breaks

1. Connection pooling — The LLM gateway keeps persistent connections. Rotating credentials means draining those pools gracefully, which none of our client libraries handle cleanly.
2. Cached query plans — Postgres caches plans per role. New credentials = new role = cold caches = latency spikes for 20 minutes.
3. Audit trails — Our compliance team requires immutable logs tying every query to a human-approvable identity. Service accounts blur that line.
4. Terraform drift — Someone manually revoked a grant in the console. Next terraform apply re-applied it. We now have a lifecycle { prevent_destroy = true } block on every DB grant resource.

The pattern that works

We treat LLM access like any other third-party integration: short-lived tokens, explicit TTL, automated rotation.

# Vault policy for LLM gateway
path "database/creds/analytics-ro" {
  capabilities = ["read"]
  ttl = "4h"
  max_ttl = "8h"
}

The gateway requests a fresh lease every 3 hours. Revocation is just TTL expiry — no manual steps, no Terraform runs. The model never sees credentials; the gateway injects them per-request.

What we'd do differently

• Never grant direct DB access. Put a read-only API layer in front. Easier to rate-limit, audit, and deprecate.
• Use workload identity federation instead of static service accounts where cloud provider supports it.
• Build the revocation path first. Before the first GRANT, write the script that removes it and test it in staging.