I am encountering an issue with the GPT-4o model deployed on Azure OpenAI Service. I have deployed the model and attempted to make chat calls using API requests and in the playground. However, as of this afternoon I consistently receive the following error message: no healthy upstream | Apim-request-id: 9243b636-765a-4b7b-9590-d7767ffef67d Here are the details of my setup: Endpoint: https://live-signals-copilot.openai.azure.com/ Pricing Tier: Standard S0 Region: East US Resource Configuration: Name: Live-Signals-Copilot Subscription: Customer Understanding Subscription ID: 62e7b93a-1bbe-492b-b06a-3a93f3d269fd Resource Group: Clarity I would like to know if this problem occurs sporadically and how to resolve it. Could you please provide guidance on how to resolve this issue and whether I should file an incident report? I see that this has been a problem for other users as well: https://learn.microsoft.com/en-us/answers/questions/2104125/how-to-resolve-openai-internalservererror-no-healt Thanks for your help.

Hi James Shaw Unhealthy stream points pointer server-side issue which indicates server is not healthy. Sometimes it is customer specific (consistent rate limits and very long queries make server unhealthy, we recommend customer to add/change deployment region, reduce query size adjust max token to by-pass existing rate limit etc.) Sometimes it is applicable to lot of customers in same region. Here in this case, It is east us. Please note that East Us and East Us 2, Sweden Central are high demand regions and face outages sometimes (Product team fixes rate limit on that server. We should load balance between multiple regions as best practice and stay attuned to Service health and Azure Status to roll out any new strategy.) Another around to change Model API version to gain more availability in the endpoint. Upon 404 error with any other non-preview API version at the endpoint URL. I think all the endpoint URLs from model deployment are showing similar syntax and looks by design (endpoint URL syntax seems to have changed c and backend implementation stays with product team only) Any modification with endpoint URL will render "resource not found error only". Please don't forget to upvote this answer if it cleared all your queries Thank you.

Failing Azure Foundry Deployment - No Healthy Upstream Error

James Shaw 0 Microsoft Employee

I am encountering an issue with the GPT-4o model deployed on Azure OpenAI Service. I have deployed the model and attempted to make chat calls using API requests and in the playground. However, as of this afternoon I consistently receive the following error message:

no healthy upstream | Apim-request-id: 9243b636-765a-4b7b-9590-d7767ffef67d

Here are the details of my setup:

Endpoint: https://live-signals-copilot.openai.azure.com/
Pricing Tier: Standard S0
Region: East US
Resource Configuration:
Name: Live-Signals-Copilot
Subscription: Customer Understanding
Subscription ID: 62e7b93a-1bbe-492b-b06a-3a93f3d269fd
- Resource Group: Clarity

I would like to know if this problem occurs sporadically and how to resolve it.

Could you please provide guidance on how to resolve this issue and whether I should file an incident report? I see that this has been a problem for other users as well:

https://learn.microsoft.com/en-us/answers/questions/2104125/how-to-resolve-openai-internalservererror-no-healt

Thanks for your help.

Manas Mohanty (Quadrant Resource LLC) 105 Reputation points Microsoft Vendor

2025-01-17T09:06:06.5533333+00:00

Hi James Shaw!

Welcome to Microsoft Q&A Forum, thank you for posting your query here!

I understand that you are encountering an issue, I just checked in east us region for any internal server error or upstream error related to GPT4o deployement. It is working fine for me at my side.

Kindly refer below screenshot.

If the issue still persists for you, please check GPT4o model deployment in other available regions or reach support ticket team if you want a detailed investigation on your issue.

kindly refer below links: azure status

Thank you.
Manas Mohanty (Quadrant Resource LLC) 105 Reputation points Microsoft Vendor

2025-01-20T04:09:56.68+00:00

Hi James Shaw!

Following up to see if the given response was helpful.

Thank you.
Manas Mohanty (Quadrant Resource LLC) 105 Reputation points Microsoft Vendor

2025-01-21T06:03:29.5166667+00:00

Hi James Shaw!

We haven’t heard from you on the last response and was just checking back to see if the give response was helpful.

Thank you.
James Shaw 0 Reputation points Microsoft Employee

2025-01-21T17:33:38.59+00:00

Hi @Manas Mohanty (Quadrant Resource LLC) , thanks for checking. I will let you know if the response was helpful
James Shaw 0 Reputation points Microsoft Employee

2025-01-22T00:01:38.32+00:00

@Manas Mohanty (Quadrant Resource LLC)

We're no longer encountering the error, and it seems to have been an intermittent issue within Azure ML. Do you know what causes this issue so we can take steps to avoid it in the future?

Also, could you explain why requests to the endpoint must use the preview API version URL parameters? I tried making a request without the parameter using and using the deployment's, non-preview version and those requests return 404's. see the api-version parameter in the url below:

https://ABC.com/openai/deployments/gpt-4o-latest/chat/completions?*api-version=2024-10-01-preview*

Thanks!
James Shaw 0 Reputation points Microsoft Employee

2025-01-22T00:04:48.4733333+00:00

We're no longer encountering the error, and it seems to have been an intermittent issue within Azure ML. Do you know what causes this issue so we can take steps to avoid it in the future?

Also, could you explain why requests to the endpoint must use the preview API version URL parameters? I tried making a request without the parameter and using a non-preview version parameter, and encounter 404 errors on these requests.

The section I am referring to is bolded in the example below:

https://endpoint/openai/deployments/gpt-4o-latest/chat/completions?api-version=2024-10-01-preview

Thanks!

1 answer

Manas Mohanty (Quadrant Resource LLC) 105 Reputation points Microsoft Vendor

2025-01-22T04:53:00.79+00:00
Hi James Shaw

Unhealthy stream points pointer server-side issue which indicates server is not healthy.

Sometimes it is customer specific (consistent rate limits and very long queries make server unhealthy, we recommend customer to add/change deployment region, reduce query size adjust max token to by-pass existing rate limit etc.)

Sometimes it is applicable to lot of customers in same region. Here in this case, It is east us. Please note that East Us and East Us 2, Sweden Central are high demand regions and face outages sometimes (Product team fixes rate limit on that server. We should load balance between multiple regions as best practice and stay attuned to Service health and Azure Status to roll out any new strategy.)

Another around to change Model API version to gain more availability in the endpoint.

Upon 404 error with any other non-preview API version at the endpoint URL. I think all the endpoint URLs from model deployment are showing similar syntax and looks by design (endpoint URL syntax seems to have changed c and backend implementation stays with product team only)

Any modification with endpoint URL will render "resource not found error only".

Please don't forget to upvote this answer if it cleared all your queries

Thank you.
Please sign in to rate this answer.
James Shaw 0 Reputation points Microsoft Employee

2025-01-22T22:57:30.2166667+00:00

Thanks Manas, we are experiencing the same issue about unhealthy upstream once again today :

Is the issue specifically related to the 08-06 API version? We would like to incorporate a feature relying on this endpoint in a way that it can handle production traffic. So what API version and region would be ideal for avoiding this problem?

Manas Mohanty (Quadrant Resource LLC) 105 Reputation points Microsoft Vendor

2025-01-23T04:24:33.0066667+00:00

Hi James Shaw,

I am also seeing "insufficient permission" error on top, please make sure you have got Cognitive Service OpenAI User or contributor permission to view and send chat completion requests.

Kindly refer

https://techcommunity.microsoft.com/blog/appsonazureblog/build-an-enterprise-ready-azure-openai-solution-with-azure-api-management/3907562

To get availability on your production app you can leverage API Managment /Application Gateway to split incoming traffic based on service health of API endpoints and load type.

Kindly refer: https://techcommunity.microsoft.com/blog/appsonazureblog/build-an-enterprise-ready-azure-openai-solution-with-azure-api-management/3907562

You can do a prior latency check on different regions by measuring chat completion time through SDK or Postman to decide your region requirements.

A pool from US and Europe / South Asian regions would be good to test with.

Note: Sweden Central, East Us, East Us2 are high demand regions

Thank you.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Failing Azure Foundry Deployment - No Healthy Upstream Error

1 answer

Your answer