Azure Container Instances Randomly Pulling and Restarting unchanged Image

Gareth Oates 6 Reputation points
2021-02-11T09:01:20.527+00:00

I received notification this morning at 08:02 that there was an availability error in my application. The App Service could not reach the container via the ip address I had configured, 10.24.2.5. I login to the portal to check and it seems the ip address of my container instances has changed to 10.24.2.4. According to the documentation

If the container group is stopped, started, or restarted, the container group's private IP is subject to change. If this happens, you will need to update the application gateway configuration.

I do not know why my container group was stopped, started or restarted, however I noticed a couple of events in the log which coincide with the time the incident occurred.

66893-pulling-events.png

This is strange to me, as there was nothing I did overnight to prompt azure container instances to try and fetch a new image. The image it's pulling is hosted in an Azure Container Registry, and from the portal we can see that this image has not changed since June 2020 and so pulling should not cause any stop/start/restart to occur.

66867-2021-02-11-09-50-21-digitalpost-activemq-microsoft.png

Can someone please explain why in the middle of the night, azure container instances takes it upon itself to try and pull a new version of my image, restarts itself and then gets a new ip address?

Azure Container Instances
Azure Container Instances
An Azure service that provides customers with a serverless container experience.
711 questions
{count} votes

12 answers

Sort by: Most helpful
  1. Darrell Burnell 1 Reputation point
    2021-02-11T11:11:12.813+00:00

    We've had the same issue but our restart policy was set to always, we are changing it to onfailure and also specifying the IP for the container instance to avoid changing IPs.

    https://learn.microsoft.com/en-us/azure/container-instances/container-instances-troubleshooting


  2. Rodrigo Espinoza 11 Reputation points
    2021-02-12T03:08:28.557+00:00

    I'm facing the same issues. We have several containers, all deployed with RestartPolicy="OnFailure", but they keep randomly pulling the image and therefore containers are restarted and most of the time changing their IP addresses. We have others processes that depend on that private IP address so if it changes, this breaks things.

    The image hasn't changed so I don't understand what else could be forcing to pull the image again.

    67160-image.png


  3. Scott Craig 1 Reputation point
    2022-01-29T09:26:06.68+00:00

    Anybody get to the bottom of why this happened ?

    0 comments No comments

  4. comray 1 Reputation point
    2022-02-09T09:09:06.363+00:00

    The same happened to our ACI setup as well, just two days ago, for unknown reasons. One container group got restarted but this activity is not shown in the activity log. We store the logs outside in a log analytics workspace so I could see the applications restart sequence as well as the uptime command showed that the container got restarted a few hours ago. I wonder what triggers this restart, so far I have not received a satisfying answer from Microsoft

    0 comments No comments

  5. Anonymous
    2022-02-10T17:39:10.763+00:00

    Same over here (unknown cause of container restart yesterday). The event log doesn't square with the container state summary in the attached screenshot (that also shows the earlier random image pulling last month). None of the container alerts were tripped either (includes start/stop/restart/start events).

    173282-image.png

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.