Rate limiting inconsistent with x-ratelimit-remaining-requests header

Question

Hi,

I've configured a gpt-4o model with a 50k tokens-per-minute limit, which Azure translates into 300 requests per minute, i.e. 50 requests for every 10-second window. 

Despite this, I hit the limit with only ten or so requests when sending them in a burst.

The issue is clear from the response headers.

Initially, the headers show x-ratelimit-remaining-requests=50 and x-ratelimit-remaining-tokens=50000. After sending 5 requests in quick succession, these values drop to x-ratelimit-remaining-requests=45 and x-ratelimit-remaining-tokens=46760. However, on the 6th request, I receive a rate limiting error, even though there should still be available capacity.

Are there undisclosed rate limits at play? What changes do you recommend?

Answer 1

Hi RXM,

Thank you for reaching out to Microsoft Q&A forum!

I understand that you're encountering inconsistent rate limiting behavior with Azure OpenAI’s GPT-4o model, where you hit the limit earlier than expected based on the x-ratelimit-remaining-requests and x-ratelimit-remaining-tokens headers.

There are multiple reasons for this issue:

While Azure OpenAI Services document rate limits in terms of per-minute and per-10-second windows, the actual enforcement may involve more granular sub-second windows (e.g., per-second or per-millisecond throttling).

This means that even if you see x-ratelimit-remaining-requests=45, the backend may apply stricter microbursts limits per second.

If you send multiple requests in quick succession, Azure might internally queue them and then reject those exceeding the instantaneous threshold. Where this can lead to situations where a request is denied even when the remaining quota appears sufficient.

And also, the headers are updated after the request is processed, meaning a burst of fast requests might cause multiple concurrent checks, leading to premature rejections.

Possible Solutions:

Use a queue-based approach where requests are processed at a controlled rate.

If you send 10 requests in 1 second, but the system allows only 5 requests per second, you’ll hit the limit even before reaching 50 in 10 seconds. Here evidence is your x-ratelimit-remaining-requests header showed 45 remaining after 5 requests, but the 6th request failed. This suggests Azure has sub-second-rate enforcement, and your burst exceeded an internal per-second cap.

Instead of relying on request count, optimize based on token consumption. If you send large prompts, reducing token usage per request may help avoid early limits.

Azure may process requests asynchronously, meaning your requests could be competing with each other. If too many requests arrive at once, some may be queued and then rejected due to exceeding instantaneous throughput. If you try sending 10 requests one at a time with 200ms gaps, it might work fine, confirming a bursting issue.

Additionally, you can refer this document (https://learn.microsoft.com/en-us/azure/ai-services/openai/quotas-limits) to avoid rate limit issues.

Hope this helps. Do let me know if you have any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful.

Thank You.