431 RequestHeaderFieldsTooLarge when calling certain Azure Open AI models via APIM

Grant Crofton 0

We're calling Azure Open AI from Power Automate to do chat completions.

Because the model we're using (gpt-35-turbo-16k 0613) will be retired soon, we're trying different models, however they all fail with 431 RequestHeaderFieldsTooLarge.

This only happens when we access Open AI via a vNet and a Private Endpoint, it works fine when we access the public endpoint directly. However we can't do this in production due to security restrictions.

I believe this is because the vNet call goes via APIM, based on the same issue reported previously in different scenarios:

https://github.com/microsoft/sample-app-aoai-chatGPT/issues/875

https://learn.microsoft.com/en-us/answers/questions/1685374/azure-ai-studio-chat-in-my-data-gpt-4o-model-reque

https://community.openai.com/t/request-header-fields-too-large/935726

https://learn.microsoft.com/en-us/answers/questions/2114269/suddenly-getting-apistatuserror-error-code-431-whe

We get this issue with the following models we've tried:

gpt-35-turbo 0125
gpt-4o-mini

I've tried different API versions including:

2024-10-21

2025-01-01-preview

As I said above we don't get the issue when using gpt-35-turbo-16k 0613. Nothing else changes between working and non-working calls other than the deployment name.

(I should point out we're not sending images, I believe this is due to APIM adding additional headers).

Khadeer Ali 2,935 Reputation points Microsoft Vendor

2025-02-03T14:58:06.3233333+00:00

@Grant Crofton ,
Welcome to Microsoft Q&A Platform!

Thanks for reaching out. To help us diagnose the issue, could you please capture and compare the request headers for both a successful call (using the public endpoint) and a failing call (using APIM and your VNet) to Azure OpenAI? Tools like your browser's developer tools (Network tab), Fiddler, or APIM's tracing can be used to capture these headers. Seeing the differences between the two sets of headers will be very helpful in identifying the problem
Khadeer Ali 2,935 Reputation points Microsoft Vendor

2025-02-04T16:08:29.29+00:00

@Grant Crofton ,

Just following up to see if you had a chance to review my previous message. To assist you effectively, providing these details will help us pinpoint the issue and offer a solution promptly. Looking forward to your response!
Grant Crofton 0 Reputation points

2025-02-04T16:26:22.4766667+00:00

Hi @Khadeer Ali , unfortunately I've not found a way to view the requests to help diagnose the issue.

The issue only happens for requests from a Power Automate Cloud Flow using a Custom connector, which go via a vNet to the Open AI Private Endpoint.

I've tried enabling vNet Flow Logs but these do not support logging traffic from Private Endpoints or Power Automate.

There's no explicit APIM instance set up in our subscription. My reason for thinking that this might be the issue was due to the URL being as follows (which I can see when using the 'test' function of the Custom connector): {guid}.10.custom.uk.azure-apihub.net/apim/enact-5ftest-20-2d-20azure-20open-20ai-5fc4d8e4e6a2c59327/{id}/openai/deployments/gpt-35-turbo-0125/chat/completions?api-version=2024-10-21. This is not a URL I've defined, the URL of the OpenAI resource is {resource name}.openai.azure.com/.

Custom connectors do not support logging of the request details.

Obviously this makes it difficult to understand what's happening - if you have any thoughts as to how I could see the request details, please let me know.

Regards,

Grant
Khadeer Ali 2,935 Reputation points Microsoft Vendor

2025-02-04T17:14:28.3733333+00:00
@Grant Crofton ,

Thanks for the extra info – that's really helpful. It sounds like we're dealing with a bit of a tricky setup with the Power Automate custom connector, the VNet, and the private endpoint. I get why you're having trouble seeing the requests; it's definitely not straightforward in this scenario.

That URL you mentioned, with the *.azure-apihub.net part, makes me think Power Automate is using some kind of internal APIM behind the scenes for the VNet connection. Is that something you've come across before? I'm wondering if that's where the header size issue might be coming from, even though we don't see a separate APIM instance in your subscription.

A couple of things we could try:

Even though the custom connector itself doesn't log requests, could you take a peek at the run history for your Power Automate flow? Sometimes, there might be some clues about the error or the data being sent, even if it's not the full request details.

If possible, would you be able to create a really simple Power Automate flow that calls the OpenAI API directly, without the custom connector? Just to see if the problem is with the connector itself, or if it's something more general with the VNet setup.

If you can call the OpenAI API directly, try using the "HTTP" action in Power Automate. It gives you a bit more control, and you could try explicitly setting the Content-Type header to application/json. Sometimes, that can help with 431 errors.

Once we try these workarounds, if we still cannot find the root cause, we can then consider how to proceed with the next steps.

Share via

431 RequestHeaderFieldsTooLarge when calling certain Azure Open AI models via APIM

Your answer