Out of quota after adding data source to model

Question

After deploying Azure OpenAI model and making test request "hello" I successfully received answer from model.
But after using "add you data" and downloading .docx document with single word "test" to blob storage I get error
"An error occurred when calling Azure OpenAI: Server responded with status 429. Error message: {'error': {'code': '429', 'message': 'Rate limit is exceeded. Try again in 10 seconds.'}}"

Answer 1

Hi @ZenbitTech ,

welcome to the Microsoft Q&A Platform!

Error you're facing—"Rate limit is exceeded. Try again in 10 seconds"—indicates that you've hit the usage limits set for your Azure OpenAI service.

• Rate Limiting: Wait 10-30 seconds before retrying or upgrade your plan.
• Quota Exceeded: Check and increase your usage limits in the Azure portal.
• Improper Document Upload: Ensure proper document upload to avoid multiple requests.
• Retry Logic: Implement retry logic with exponential backoff to handle rate limits.
• Scaling Resources: Scale your Azure OpenAI instance to handle higher usage. If the answer is helpful, please click "Accept Answer" and kindly upvote it.

Answer 2

Try requesting for more TPM. Preferably, over 100K. I had mine at 120TPM but requested for 200TPM. I appreciated you reaching out to me. Cheers!

Answer 3

It seems like you have exceeded the rate limit for Azure OpenAI service. The error message suggests that you should try again in 10 seconds. This error occurs when the number of requests per second reaches the limit of managed online endpoints. To mitigate this issue, you can implement retry logic in your application, avoid sharp changes in the workload, increase the workload gradually, test different load increase patterns, and create another OpenAI service resource in the same or different regions, and distribute the workload among them. If you need to increase the limit, you can raise request for a quota increase