Random Instance Errors on Azure Function App (Python) - Elastic Plan

Question

Hi all,

After switching the ASP we use for our function app to Elastic Premium, we are having issues with the uptime of our app. I will provide a list of details below and things we checked and tried.

Info about the app:

• It hosts our public facing API
• Python 3.11
• Elastic Premium (EP1)
• Always On is turned OFF
• 1 always ready instance
• Requests come in through JWT verified calls in API Management
• Health Check setup and uses a custom endpoint with all client connections

What happens:

The instances sometimes fail, but we have so far been unable to find out why. The Microsoft docs suggest always having multiple instances running to mitigate having a single point of failure which we will also do, but we first want to find the root cause.

Logs:

In the Application Insights we see RpcExceptions, TokenExpired and ClientConnectionFailures, which are all errors that can only be thrown when the instance is running so these are not it.

Then we dove deeper and looked into Kudu logs which show the following on a failing instance:
ERROR - Container func-###-###-api_0_a244931d for site func-###-###-api has exited, failing site start

So it appears sometimes containers fail to start, but... it doesn't show why.

All the Kudu logs show this line:
2025-01-08T14:15:29.492Z INFO - Logging is not enabled for this container.

Is it possible to enable container logging? And will it make a difference if we have it?

The total Kudu logs of the containers are ~30 lines of normal bootup info (image up to date, pulling image etc.), failing containers only appear to happen when the API is being used, but not always. We track what users do through LogRocket and when we replicate their exact steps we have never been able to replicate a container failure.

If anyone could help us in the direction to find the root cause of this we would greatly appreciate it. If specifics logs can help find the origin also please let me know so I can provide them.