The rate-limit counter on my gpt-5-mini deployment decrements by

Riley82 Advanced 49m ago 303 views 5 likes 2 min read

Here's what I did: I logged x-ratelimit-remaining-tokens on a Global Standard gpt-5-mini (1M TPM, Sweden Central, API version 2024-12-01-preview), computed each charge as limit - remaining, and kept only responses where a single request finished within ±2 seconds so the delta was attributable to one call.

Breakdown of the 29 paired samples:

  • 20 charged exactly usage.prompt_tokens (examples: 8,329 / 13,605 / 40,073 / 93,837)
  • 2 within 5% of input + max_tokens or max(input, max_tokens) — probably coincidence
  • 7 charged well above input — all retried after connection errors (one request with 252,781 input tokens was charged 630,582, suggesting two attempts each hit by a character-based input estimate)

This rules out the documented formulas:

  • One request was charged 8,329 tokens with max_completion_tokens = 10000. No formula involving max_tokens — sum or maximum — can produce a charge below max_tokens itself.
  • input + completion matched 0 of 29. Completions aren't counted.
  • Behavior was identical with reasoning_effort at minimal, low, and unset. Not a reasoning-budget effect.

The Azure docs are contradictory. One section says the estimate "includes prompt text and count, the max_tokens parameter setting, the best_of parameter setting" — a worst-case ceiling reading. The 429 section commits to an actual sum: "prompt + max_tokens." Neither matches reality. OpenAI's platform docs use max(max_tokens, character-based estimate) — also doesn't match.

So for a completed request, the counter visibly ends at exact input tokens with no max_tokens component. What I can't observe externally:

1. Is the counter charged when the request arrives (the documented prompt + max_tokens estimate, later reconciled to actual input), or charged once at completion with the exact input?
2. If requests are charged on receipt, does the over-estimate get refunded, or is the refund delayed/aggregated such that it's invisible in my per-request delta?
3. For the retried requests, is each attempt charged independently against the remaining budget, or is there a deduplication window?

Has anyone else seen this exact-input-only behavior? I'm especially curious if the charging-on-receipt theory holds — that would explain why max_tokens never shows up in completed-request deltas.

Help Wanted

All Replies (4)

D
DrewCrafter Novice 45m ago
I noticed adding a small buffer to that math helps avoid those unexpected 429 spikes.
0 Reply
M
MaxOwl Intermediate 43m ago
Did you factor in the overhead for system prompts, or just the user input tokens?
0 Reply
M
MaxWhiz Expert 39m ago
@MaxOwl Imagine thinking I’d forget the system prompt, I'm not a complete amateur lol
0 Reply
R
RayTinkerer Novice 41m ago
Same thing happened to me. I just bake in a 5% margin to be safe.
0 Reply

Write a Reply

Markdown supported