Slow and unstable responses from o1

Josh Sirota 40

As of February 18, 2025, I'm running into issues with o1 on Azure AI services that I wasn't previously.

When running o1 on a series of inputs, response times are slower than I saw before (500 queries taking ~1.5 hours instead of 15 minutes seen previously), and eventually the service chokes entirely and stops returning responses. I've seen openai.TimeoutErrors pop up as well.

Vikram Singh 1,980 Reputation points Microsoft Employee

2025-02-21T09:20:22.2733333+00:00

Hi Josh Sirota,

Thanks for posting your question in Microsoft Q&A, apologize for any inconvenience caused on this. To assist you better, could you please replicate the issue and share api details, the latest error, including the full error message, apim-request-id, timestamp when the errors occurred.

This information will help us investigate the issue more effectively.

Thank you for your cooperation!
Vikram Singh 1,980 Reputation points Microsoft Employee

2025-02-26T06:52:47.49+00:00

Hi Josh Sirota,

Please confirm if you are in EastUS2 region or SwedenCentral region? If yes, there is an ongoing issue which caused an unexpected latency for response.

I will update you the status here daily until this issue has been mitigated, as of today, there is no ETA, but there are several tickets opening to product team and product team is investigating this issue.

I hope this helps.
Vikram Singh 1,980 Reputation points Microsoft Employee

2025-02-28T06:24:46.3066667+00:00

Hi Josh Sirota,

Thank you for your patience. Latest update and for EastUS2 region -

Since this issue was caused by limit capacity, the product team update the design to enable dynamic routing which will solve time out issues.

The dynamic routing is built and will be release step by step, hope to cover most of the customers by the end of this week.

There are some options for now, one is Provisioned Managed sku for latency sensitive workload, and you may want to switch to other regions as alternative plan for this week.

Please refer to this document to Provisioned Managed sku - https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/provisioned-throughput-onboarding

Other regions/version for o1 -

https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=global-standard%2Cstandard-chat-completions#global-standard-model-availability

I will update here again once this issue confirmed resolved, will let you know more.

I hope this helps.

Share via

Slow and unstable responses from o1

Your answer