431 RequestHeaderFieldsTooLarge when calling certain Azure Open AI models from Power Automate

Question

431 RequestHeaderFieldsTooLarge when calling certain Azure Open AI models from Power Automate

Grant Crofton 20

We're calling Azure Open AI from Power Automate to do chat completions.

Because the model we're using (gpt-35-turbo-16k 0613) will be retired soon, we're trying different models, however they all fail with 431 RequestHeaderFieldsTooLarge.

This only happens when we access Open AI via a vNet and a Private Endpoint, it works fine when we access the public endpoint directly. However we can't do this in production due to security restrictions.

I believe this is because the vNet call goes via APIM, based on the same issue reported previously in different scenarios:

https://github.com/microsoft/sample-app-aoai-chatGPT/issues/875

https://learn.microsoft.com/en-us/answers/questions/1685374/azure-ai-studio-chat-in-my-data-gpt-4o-model-reque

https://community.openai.com/t/request-header-fields-too-large/935726

https://learn.microsoft.com/en-us/answers/questions/2114269/suddenly-getting-apistatuserror-error-code-431-whe

We get this issue with the following models we've tried:

gpt-35-turbo 0125
gpt-4o-mini

I've tried different API versions including:

2024-10-21

2025-01-01-preview

As I said above we don't get the issue when using gpt-35-turbo-16k 0613. Nothing else changes between working and non-working calls other than the deployment name.

(I should point out we're not sending images, I believe this is due to APIM adding additional headers).

Khadeer Ali 4,235 Reputation points Microsoft External Staff

2025-02-03T14:58:06.3233333+00:00

@Grant Crofton ,
Welcome to Microsoft Q&A Platform!

Thanks for reaching out. To help us diagnose the issue, could you please capture and compare the request headers for both a successful call (using the public endpoint) and a failing call (using APIM and your VNet) to Azure OpenAI? Tools like your browser's developer tools (Network tab), Fiddler, or APIM's tracing can be used to capture these headers. Seeing the differences between the two sets of headers will be very helpful in identifying the problem
Khadeer Ali 4,235 Reputation points Microsoft External Staff

2025-02-04T16:08:29.29+00:00

@Grant Crofton ,

Just following up to see if you had a chance to review my previous message. To assist you effectively, providing these details will help us pinpoint the issue and offer a solution promptly. Looking forward to your response!
Grant Crofton 20 Reputation points

2025-02-04T16:26:22.4766667+00:00

Hi @Khadeer Ali , unfortunately I've not found a way to view the requests to help diagnose the issue.

The issue only happens for requests from a Power Automate Cloud Flow using a Custom connector, which go via a vNet to the Open AI Private Endpoint.

I've tried enabling vNet Flow Logs but these do not support logging traffic from Private Endpoints or Power Automate.

There's no explicit APIM instance set up in our subscription. My reason for thinking that this might be the issue was due to the URL being as follows (which I can see when using the 'test' function of the Custom connector): {guid}.10.custom.uk.azure-apihub.net/apim/enact-5ftest-20-2d-20azure-20open-20ai-5fc4d8e4e6a2c59327/{id}/openai/deployments/gpt-35-turbo-0125/chat/completions?api-version=2024-10-21. This is not a URL I've defined, the URL of the OpenAI resource is {resource name}.openai.azure.com/.

Custom connectors do not support logging of the request details.

Obviously this makes it difficult to understand what's happening - if you have any thoughts as to how I could see the request details, please let me know.

Regards,

Grant
Khadeer Ali 4,235 Reputation points Microsoft External Staff

2025-02-04T17:14:28.3733333+00:00
@Grant Crofton ,

Thanks for the extra info – that's really helpful. It sounds like we're dealing with a bit of a tricky setup with the Power Automate custom connector, the VNet, and the private endpoint. I get why you're having trouble seeing the requests; it's definitely not straightforward in this scenario.

That URL you mentioned, with the *.azure-apihub.net part, makes me think Power Automate is using some kind of internal APIM behind the scenes for the VNet connection. Is that something you've come across before? I'm wondering if that's where the header size issue might be coming from, even though we don't see a separate APIM instance in your subscription.

A couple of things we could try:

Even though the custom connector itself doesn't log requests, could you take a peek at the run history for your Power Automate flow? Sometimes, there might be some clues about the error or the data being sent, even if it's not the full request details.

If possible, would you be able to create a really simple Power Automate flow that calls the OpenAI API directly, without the custom connector? Just to see if the problem is with the connector itself, or if it's something more general with the VNet setup.

If you can call the OpenAI API directly, try using the "HTTP" action in Power Automate. It gives you a bit more control, and you could try explicitly setting the Content-Type header to application/json. Sometimes, that can help with 431 errors.

Once we try these workarounds, if we still cannot find the root cause, we can then consider how to proceed with the next steps.
Grant Crofton 20 Reputation points

2025-02-05T16:34:06.36+00:00
Thanks Khadeer.

To clarify, the APIM endpoint I see is when I use the Test feature of the Custom connector, which sends a request from the browser. I'm not sure if this is used when the Cloud Flow runs. Another important point is that this test succeeds - it's only when run under the context of the Cloud Flow that we get the 431.

There's nothing revealing in the run history of the cloud flows - the only different between successful and unsuccessful runs is the deployment name we pass (or whether public network access is enabled, but that's configured in Azure).

The OpenAI call works when we do it directly, also it works when we do it via an HTTP connector (without vNet as this is not supported by HTTP connector), it also works when we use the Custom connector without vNet. It also works with the Custom connector with vNet when we use the gpt-35-turbo-16k 0613 model.

The only situation in which it doesn't work is:

Open AI with Private Endpoint

Virtual Network support enabled in Power Automate via Enterprise Policy

Custom connector

Model is gpt-35-turbo 0125 or gpt-4o-mini (maybe some others, we only tried these)

Unfortunately that's the configuration we need to use, as the gpt-35-turbo-16k 0613 model which does work is being retired next week, and our security standards require we have public network access disabled.

One interesting aspect about the models which do and don't work is that the ones that don't work are more recent deployments (which we configure in AI Foundry). It could be that the issue is related to the timing of when we created the deployments, rather than which specific model is used.
Grant Crofton 20 Reputation points

2025-02-05T16:38:25.51+00:00

[duplicate]
santoshkc 13,600 Reputation points Microsoft External Staff

2025-02-11T10:53:22.43+00:00

Hi @Grant Crofton,

Apologies for the delay in response. The 431 error is likely caused by additional headers being added when routing through APIM via a Private Endpoint, exceeding the allowed header size. Since older deployments work while newer ones fail, the issue may stem from how these models interact with APIM. One possible workaround is to reduce header size if configurable in APIM. Check if unnecessary headers are being added in the Custom Connector settings and remove any that are not required. If APIM policies are in place, consider modifying them to limit the size of headers being forwarded.

Additionally, since Power Automate’s Custom Connector does not provide detailed logging, you may need to use Application Insights or diagnostic logs on the Private Endpoint to analyse the request headers.

Thank you.
santoshkc 13,600 Reputation points Microsoft External Staff

2025-02-12T15:37:47.7633333+00:00

Hi Grant Crofton,

Following up to see if the given response was helpful. Thank you.

1 answer

Your answer

Khadeer Ali 4,235 Reputation points Microsoft External Staff

2025-02-03T14:58:06.3233333+00:00

@Grant Crofton ,
Welcome to Microsoft Q&A Platform!

Thanks for reaching out. To help us diagnose the issue, could you please capture and compare the request headers for both a successful call (using the public endpoint) and a failing call (using APIM and your VNet) to Azure OpenAI? Tools like your browser's developer tools (Network tab), Fiddler, or APIM's tracing can be used to capture these headers. Seeing the differences between the two sets of headers will be very helpful in identifying the problem
Khadeer Ali 4,235 Reputation points Microsoft External Staff

2025-02-04T16:08:29.29+00:00

@Grant Crofton ,

Just following up to see if you had a chance to review my previous message. To assist you effectively, providing these details will help us pinpoint the issue and offer a solution promptly. Looking forward to your response!
Grant Crofton 20 Reputation points

2025-02-04T16:26:22.4766667+00:00

Hi @Khadeer Ali , unfortunately I've not found a way to view the requests to help diagnose the issue.

The issue only happens for requests from a Power Automate Cloud Flow using a Custom connector, which go via a vNet to the Open AI Private Endpoint.

I've tried enabling vNet Flow Logs but these do not support logging traffic from Private Endpoints or Power Automate.

There's no explicit APIM instance set up in our subscription. My reason for thinking that this might be the issue was due to the URL being as follows (which I can see when using the 'test' function of the Custom connector): {guid}.10.custom.uk.azure-apihub.net/apim/enact-5ftest-20-2d-20azure-20open-20ai-5fc4d8e4e6a2c59327/{id}/openai/deployments/gpt-35-turbo-0125/chat/completions?api-version=2024-10-21. This is not a URL I've defined, the URL of the OpenAI resource is {resource name}.openai.azure.com/.

Custom connectors do not support logging of the request details.

Obviously this makes it difficult to understand what's happening - if you have any thoughts as to how I could see the request details, please let me know.

Regards,

Grant
Khadeer Ali 4,235 Reputation points Microsoft External Staff

2025-02-04T17:14:28.3733333+00:00

@Grant Crofton ,

Thanks for the extra info – that's really helpful. It sounds like we're dealing with a bit of a tricky setup with the Power Automate custom connector, the VNet, and the private endpoint. I get why you're having trouble seeing the requests; it's definitely not straightforward in this scenario.

That URL you mentioned, with the *.azure-apihub.net part, makes me think Power Automate is using some kind of internal APIM behind the scenes for the VNet connection. Is that something you've come across before? I'm wondering if that's where the header size issue might be coming from, even though we don't see a separate APIM instance in your subscription.

A couple of things we could try:

Even though the custom connector itself doesn't log requests, could you take a peek at the run history for your Power Automate flow? Sometimes, there might be some clues about the error or the data being sent, even if it's not the full request details.

If possible, would you be able to create a really simple Power Automate flow that calls the OpenAI API directly, without the custom connector? Just to see if the problem is with the connector itself, or if it's something more general with the VNet setup.

If you can call the OpenAI API directly, try using the "HTTP" action in Power Automate. It gives you a bit more control, and you could try explicitly setting the Content-Type header to application/json. Sometimes, that can help with 431 errors.

Once we try these workarounds, if we still cannot find the root cause, we can then consider how to proceed with the next steps.
Grant Crofton 20 Reputation points

2025-02-05T16:34:06.36+00:00

Thanks Khadeer.

To clarify, the APIM endpoint I see is when I use the Test feature of the Custom connector, which sends a request from the browser. I'm not sure if this is used when the Cloud Flow runs. Another important point is that this test succeeds - it's only when run under the context of the Cloud Flow that we get the 431.

There's nothing revealing in the run history of the cloud flows - the only different between successful and unsuccessful runs is the deployment name we pass (or whether public network access is enabled, but that's configured in Azure).

The OpenAI call works when we do it directly, also it works when we do it via an HTTP connector (without vNet as this is not supported by HTTP connector), it also works when we use the Custom connector without vNet. It also works with the Custom connector with vNet when we use the gpt-35-turbo-16k 0613 model.

The only situation in which it doesn't work is:

Open AI with Private Endpoint

Virtual Network support enabled in Power Automate via Enterprise Policy

Custom connector

Model is gpt-35-turbo 0125 or gpt-4o-mini (maybe some others, we only tried these)

Unfortunately that's the configuration we need to use, as the gpt-35-turbo-16k 0613 model which does work is being retired next week, and our security standards require we have public network access disabled.

One interesting aspect about the models which do and don't work is that the ones that don't work are more recent deployments (which we configure in AI Foundry). It could be that the issue is related to the timing of when we created the deployments, rather than which specific model is used.
Grant Crofton 20 Reputation points

2025-02-05T16:38:25.51+00:00

[duplicate]
santoshkc 13,600 Reputation points Microsoft External Staff

2025-02-11T10:53:22.43+00:00

Hi @Grant Crofton,

Apologies for the delay in response. The 431 error is likely caused by additional headers being added when routing through APIM via a Private Endpoint, exceeding the allowed header size. Since older deployments work while newer ones fail, the issue may stem from how these models interact with APIM. One possible workaround is to reduce header size if configurable in APIM. Check if unnecessary headers are being added in the Custom Connector settings and remove any that are not required. If APIM policies are in place, consider modifying them to limit the size of headers being forwarded.

Additionally, since Power Automate’s Custom Connector does not provide detailed logging, you may need to use Application Insights or diagnostic logs on the Private Endpoint to analyse the request headers.

Thank you.
santoshkc 13,600 Reputation points Microsoft External Staff

2025-02-12T15:37:47.7633333+00:00

Hi Grant Crofton,

Following up to see if the given response was helpful. Thank you.

Answer 1

Grant Crofton 20

Hi @santoshkc , thanks for your reply.

To clarify, we don't have an APIM instance, this is something used behind the scenes as part of the Custom connector, as shown here: https://learn.microsoft.com/en-us/connectors/connector-architecture#architecture-components

So we don't have any control over that. We're not specifying any headers as part of the Custom connector configuration either.

I've not found a way of logging the HTTP requests - there's vNet Flow Logs but these don't work on a Private Endpoint. I've got Diagnostic Logs on the vNet and the Network Interface but these don't seem to log the HTTP requests. If you know of a way to log these let me know.

The only thing I can think of is to create an APIM instance and route the requests through that, which would enable logging of the requests and stripping out any unwanted headers.

santoshkc 13,600 Reputation points Microsoft External Staff

2025-02-13T14:39:21.97+00:00

Hi Grant Crofton,

Since APIM is managed behind the scenes in Power Automate’s Custom Connector, you don’t have direct control over it. Logging requests through a Private Endpoint is tricky, but setting up your own APIM instance could help track and filter out any unnecessary headers. Another option is to compare headers from a direct API call (like in Postman) with the failing request to spot differences.

Thank you.

Share via

431 RequestHeaderFieldsTooLarge when calling certain Azure Open AI models from Power Automate

1 answer

Your answer