Container in Azure Functions does not emit logs and stops unexpectedly, but works locally

Question

Hello,

I am experiencing an issue with an Azure Function running in a container. The container works perfectly when run locally, but fails to start after deployment to Azure.

The function includes a Python script that connects to Azure OpenAI, which runs without any issues locally. However, after deployment, the container does not start, and I am unable to debug it because the container does not emit any logs. Instead, I only see Azure logs attempting to reach the container for 300 seconds. App Insights does not capture any traces, and the log stream only shows these connection attempts.

Through extensive troubleshooting, I identified a specific line in the script that seems to trigger the issue. This line is straightforward and should not cause any issues remotely, especially as it works fine locally. Thus, my main concern is with the lack of container logs, which are essential for further diagnosis.

Could you advise on how I can access the logs generated by the container itself to troubleshoot this issue?

Thank you for your assistance.

Settings:

FUNCTIONS_WORKER_RUNTIME = python

FUNCTIONS_EXTENSION_VERSION=4

The log from logstream:

2024-10-28T14:59:45.372Z INFO  - Pulling image: mcr.microsoft.com/appsvc/msitokenservice:stage5
2024-10-28T14:59:45.609Z INFO  - stage5 Pulling from appsvc/msitokenservice
2024-10-28T14:59:45.630Z INFO  -  Digest: sha256:15c4360552ca5d886db18e4eaa6a95993305c67dae381aa8505592cd7c45bd46
2024-10-28T14:59:45.638Z INFO  -  Status: Image is up to date for mcr.microsoft.com/appsvc/msitokenservice:stage5
2024-10-28T14:59:45.652Z INFO  - Pull Image successful, Time taken: 0 Seconds
2024-10-28T14:59:45.754Z INFO  - Starting container for site
2024-10-28T14:59:45.755Z INFO  - docker run -d --expose=8081 --name <NAME>_0_5365f898_msiProxy -e WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_HOSTNAME=<NAME>.azurewebsites.net -e WEBSITE_INSTANCE_ID=6eff5a7306b63169d8544581ad99a4f06cd7b14b38e0d6d912156acbe82fa139 -e HTTP_LOGGING_ENABLED=1 mcr.microsoft.com/appsvc/msitokenservice:stage5 
1ad99a4f06cd7b14b38e0d6d912156acbe82fa139 -e HTTP_LOGGING_ENABLED=1 <NAME>sacr.azurecr.io/<NAME>s:4708 
2024-10-28T15:00:17.964Z INFO  - Initiating warmup request to container <NAME>_0_5365f898_msiProxy for site <NAME>
2024-10-28T15:00:18.036Z INFO  - Container <NAME>_0_5365f898_msiProxy for site <NAME> initialized successfully and is ready to serve requests.
2024-10-28T15:00:18.043Z INFO  - Initiating warmup request to container <NAME>_0_5365f898 for site <NAME>
2024-10-28T15:00:51.033Z INFO  - Waiting for response to warmup request for container <NAME>_0_5365f898. Elapsed time = 33.087263 sec
2024-10-28T15:01:08.562Z INFO  - Waiting for response to warmup request for container <NAME>_0_5365f898. Elapsed time = 50.6159878 sec
2024-10-28T15:01:25.787Z INFO  - Waiting for response to warmup request for container <NAME>_0_5365f898. Elapsed time = 67.8415137 sec
2024-10-28T15:01:44.045Z INFO  - Waiting for response to warmup request for container <NAME>_0_5365f898. Elapsed time = 86.0989426 sec
2024-10-28T15:02:00.738Z INFO  - Waiting for response to warmup request for container <NAME>_0_5365f898. Elapsed time = 102.7926139 sec
2024-10-28T15:02:20.170Z INFO  - Waiting for response to warmup request for container <NAME>_0_5365f898. Elapsed time = 122.2240945 sec
2024-10-28T15:02:39.941Z INFO  - Waiting for response to warmup request for container <NAME>_0_5365f898. Elapsed time = 141.9954746 sec
2024-10-28T15:02:59.395Z INFO  - Waiting for response to warmup request for container <NAME>_0_5365f898. Elapsed time = 161.4488913 sec
2024-10-28T15:03:15.765Z INFO  - Waiting for response to warmup request for container <NAME>_0_5365f898. Elapsed time = 177.8195102 sec
2024-10-28T15:03:32.991Z INFO  - Waiting for response to warmup request for container <NAME>_0_5365f898. Elapsed time = 195.0451201 sec
2024-10-28T15:03:50.252Z INFO  - Waiting for response to warmup request for container <NAME>_0_5365f898. Elapsed time = 212.3063835 sec
2024-10-28T15:04:06.713Z INFO  - Waiting for response to warmup request for container <NAME>_0_5365f898. Elapsed time = 228.7676738 sec
2024-10-28T15:04:09.174Z ERROR - Container <NAME>_0_5365f898 for site <NAME> did not start within expected time limit. Elapsed time = 231.2285258 sec
2024-10-28T15:04:09.265Z ERROR - Container <NAME>_0_5365f898 didn't respond to HTTP pings on port: 80, failing site start. See container logs for debugging.
2024-10-28T15:04:09.507Z INFO  - Stopping site <NAME> because it failed during startup.

Answer 1

Hello! @Johannes R

Based on the logs you provided, it seems that the container is not starting up correctly, and the logs are not being emitted. This can make it difficult to diagnose the issue. One thing you can try is to enable container logging in your Azure Function. This will allow you to see the logs generated by the container itself, which may help you diagnose the issue. To enable container logging, you can follow these 7 steps:

1. Go to your Function App in the Azure portal.
2. Click on "Platform features" in the left-hand menu.
3. Click on "Container settings".
4. Under "Diagnostic logs", click "On".
5. Select "Container" as the log type.
6. Choose a retention period for the logs.
7. Click "Save". Once you have enabled container logging, you can try redeploying your container and see if any logs are emitted. If you still don't see any logs, you may need to investigate further to determine why the container is not starting up correctly.

Another thing you can try is to SSH into the container and manually start the Python script to see if any errors are emitted. To do this, you can follow these 7 steps:

1. Go to your Function App in the Azure portal.
2. Click on "Platform features" in the left-hand menu.
3. Click on "SSH".
4. Run the command docker ps to get the container ID.
5. Run the command docker exec -it /bin/bash to SSH into the container.
6. Navigate to the directory where your Python script is located.
7. Run the Python script manually and see if any errors are emitted.

---

I hope that this response has addressed your query and helped you overcome your challenges. If so, please mark this response as Answered. This will not only acknowledge our efforts, but also assist other community members who may be looking for similar solutions.

Answer 2

We fixed this by importing the package not at the top of the file but within the function definition like this:

def some_function():
    from package import class
    obj = class()
    return obj