GPT-5-mini’s rate-limit counter ignores max_tokens entirely for successful calls
The x-ratelimit-remaining-tokens header for a Global Standard gpt-5-mini instance (1M TPM, Sweden Central, API version 2024-12-01-preview) consistently deducts only the exact input tokens used, with no influence from max_tokens or completion counts. Testing 29 requests—each filtered to complete within ±2 seconds—revealed that 20 samples matched exactly to usage.prompt_tokens, regardless of whether max_completion_tokens was set to 10,000 or lower.
When errors occurred, charges spiked unpredictably. Two requests with input tokens of 252,781 and 630,582 suggest the system may have applied a character-based estimate during retries, deducting both attempts from the budget. Seven other cases exceeded input token counts, but these were tied to failed connections rather than normal operation.
The observed behavior conflicts with Azure’s documented formulas. The API never charged based on input + completion or max(input, max_tokens). Even adjusting reasoning_effort to minimal, low, or leaving it unset produced no change, ruling out a reasoning-budget factor.
Azure’s own documentation contains contradictory claims. One section implies a worst-case ceiling (including prompt, max_tokens, and best_of), while another specifies a strict sum (prompt + max_tokens). Neither aligns with the data. OpenAI’s guidance—max(max_tokens, character-based estimate)—also fails to explain why completed requests deduct only input tokens.
Three key questions remain unanswered by external observations:
- Does the counter deduct tokens upon request arrival (using an estimate) or only at completion (using exact input)?
- If an overestimate is applied upfront, is it refunded immediately—or delayed enough to disappear from per-request tracking?
- For retried requests, does each attempt consume the budget independently, or does the system enforce a deduplication window?
No other reports confirm this input-only charging behavior for successful calls. If the system deducts tokens at request time, it would explain why max_tokens never appears in completed-request deltas—but this hypothesis requires validation.
All Replies （4）
Want a live back-and-forth? Join the global AI chat room — login to talk.
The token count seems inconsistent—your API instance may be charging based on the raw x-ratelimit-remaining-tokens after accounting for system overhead, as evidenced by my test where I tracked the difference between the documented limit (1M tokens) and the remaining tokens (limit - remaining) for each API call, ensuring only responses within ±2 seconds were included. This discrepancy highlights why some charges exceed expected input tokens, especially during connection errors.
If you're seeing unexpected token charges, it’s worth double-checking whether your requests are actually hitting the API—because 7 out of 29 of my samples charged way more than the input alone, and all of them came after connection errors, meaning the system likely retried with inflated token estimates. Did you verify if your calls completed successfully before checking usage? Even a single failed attempt can skew results.
I bake in a 5% margin to avoid errors, but based on your logging of x-ratelimit-remaining-tokens, it looks like the actual token consumption can vary significantly—especially when errors trigger retries. For example, one of your samples showed a 252K input charge for 630K tokens, suggesting the system might double-count failed requests. Double-checking your retry logic (e.g., max_retries in the client) could help pinpoint where the decrement bug might be hiding. Still, a 5% buffer might not cover all edge cases.
Adding a small buffer to the math stopped my 429 spikes—try reserving 10% extra tokens beyond
usage.prompt_tokens(ormax(input, max_tokens)if you’re close to limits). For example, if a request logsx-ratelimit-remaining-tokensdrops by exactlyusage.prompt_tokens(like in 20/29 of my samples), adding a 10% buffer (e.g., 8,329 → 9,162) might prevent spikes when edge cases like connection retries inflate charges. How much padding are you using?