VMSS stopped suddenly doing AutoOSUpgrade

Wagholikar, Neha 5

We are 3 VMSS in each env, in all env suddenly VMSS has stopped doing AutoOSUpgrade even after 12hrs of image creation.

last AutoOSUpgrade was done at oct25

Upgrade Policy mode : Rolling

Mahesh Goud Juvvadi 1,665 Reputation points Microsoft Vendor

2024-11-07T18:58:23.3633333+00:00

Hi Wagholikar, Neha,

Welcome to the Microsoft Q&A Platform. Thank you for posting your query here.

Based upon your query, you are encountering an issue where the Virtual Machine Scale Sets have stopped performing AutoOSUpgrade.

Make sure that the upgrade policy mode for your VMSS is configured correctly. For AutoOSUpgrade to function properly, the policy mode should be set to "Automatic." In some cases, upgrades may be pending due to unmet prerequisites. Check if there are any pending updates or other prerequisites that need to be completed for the upgrade to proceed.

Additionally, review the VMSS and Azure Activity logs for any error messages or warnings that could provide insight into why the upgrade is not progressing. These logs can provide information to help identify the underlying issue.

Please find the below document for your reference:

https://learn.microsoft.com/en-us/azure/virtual-machine-scale-sets/virtual-machine-scale-sets-automatic-upgrade

If you find this information helpful, please consider clicking "Upvote" on the post.

Thank you.

Wagholikar, Neha 5

Hi @Mahesh Goud Juvvadi ,

AutoOSUpgrade is set to 'Rolling' mode

below is json of existing vmss

also in logs 'AutoOsUpgrade' there are no logs

System is Windows 2022 Datacenter and other is Rhel8

"upgradePolicy": {

        "mode": "Rolling",

        "rollingUpgradePolicy": {

            "maxBatchInstancePercent": 20,

            "maxUnhealthyInstancePercent": 20,

            "maxUnhealthyUpgradedInstancePercent": 20,

            "pauseTimeBetweenBatches": "PT0S",

            "maxSurge": false,

            "rollbackFailedInstancesOnPolicyBreach": false

        },

        "automaticOSUpgradePolicy": {

            "enableAutomaticOSUpgrade": true,

            "useRollingUpgradePolicy": false,

            "disableAutomaticRollback": false

        }

    },

Wagholikar, Neha 5 Reputation points

2024-11-08T14:11:46.8566667+00:00

also in Image Reference of VMSS under Operating System section , should it contain '<subscriptionid>/versions/latest' ?
Mahesh Goud Juvvadi 1,665 Reputation points Microsoft Vendor

2024-11-08T18:06:17.31+00:00

Hi Wagholikar, Neha,

Based upon your recent response we can suggest you make sure the rolling upgrade policy is used for automatic OS upgrades in your VMSS, set "useRollingUpgradePolicy": true.

This will apply the rolling upgrade policy (like upgrading instances in batches) during OS upgrades. If it's set to false, Azure won't use the rolling upgrade process and might upgrade all instances at once, which may not be what you want.

Thank you.
Wagholikar, Neha 5 Reputation points

2024-11-08T19:00:04.7233333+00:00

Hi

I have tried that too, for 3hrs still the OS upgrade was not started
Srinud 3,400 Reputation points Microsoft Vendor

2024-11-12T05:42:02.5133333+00:00
Hi Wagholikar, Neha,
Thank you for sharing update on the issue!

"properties": { "upgradePolicy": { "mode": "Automatic", "RollingUpgradePolicy": { "BatchInstancePercent": 20, "MaxUnhealthyInstancePercent": 25, "MaxUnhealthyUpgradedInstancePercent": 25, "PauseTimeBetweenBatches": "PT0S" }, "automaticOSUpgradePolicy": { "enableAutomaticOSUpgrade": true, "useRollingUpgradePolicy": true, "disableAutomaticRollback": false

Set upgradepolicy mode as "Automatic" and use "useRollingUpgradepolicy" is true then aplly the changes.
Please find the below document for more information: https://learn.microsoft.com/en-us/azure/virtual-machine-scale-sets/virtual-machine-scale-sets-automatic-upgrade

If you find this information helpful, please consider clicking "Upvote" on the post.
Wagholikar, Neha 5 Reputation points

2024-11-12T09:12:20.9433333+00:00

Hi @Srinud ,

Actually we require 'Rolling' mode in upgrade policy.
Previously it was working fine AutoOsUpgrade was getting started automatically, suddenly it has been stopped from Oct mid
Srinud 3,400 Reputation points Microsoft Vendor

2024-11-13T19:19:10.1033333+00:00

Hi Wagholikar, Neha,

Rolling upgrade policy and MaxSurge for Virtual Machine scale Sets with Flexible Orchestration is currently in preview.

Please find the below document for more information: https://learn.microsoft.com/en-us/azure/virtual-machine-scale-sets/virtual-machine-scale-sets-configure-rolling-upgrades?source=recommendations&tabs=portal1%2Cportal2%2Cportal3%2Ccli4

Thank you.
Wagholikar, Neha 5 Reputation points

2024-11-14T06:28:48.8233333+00:00

Hi @Srinud ,

Rolling upgrade policy for Virtual Machine Scale sets with Uniform Orchestration is in general availability (GA).

**MaxSurge for Virtual Machine scale Sets with Uniform Orchestration is currently in preview.

'MaxSurge' for Uniform Orchestration is currently in preview

but Rolling Upgrade is in GA and we are using it from long time.

Issue is after enabling rolling upgrade and 'enableAutomaticOSUpgrade: true' it usually starts automatically but from last 1 month its not starting**
Wagholikar, Neha 5 Reputation points

2024-11-14T18:12:43.31+00:00

Hi @Srinud

As per notes in given link :
Rolling upgrade policy for Virtual Machine Scale sets with Uniform Orchestration is in general availability (GA).

MaxSurge for Virtual Machine scale Sets with Uniform Orchestration is currently in preview.

**Rolling upgrade policy and MaxSurge for Virtual Machine scale Sets with Flexible Orchestration is currently in preview.

Rolling Upgrade is in GA
Also we have are using Rolling Upgrade from long time , now suddenly it has been stopped working to do Automatic Updates (meaning taking latest image from compute gallery applying it)**
Srinud 3,400 Reputation points Microsoft Vendor

2024-11-15T10:01:58.2333333+00:00

Hi Wagholikar, Neha,
Thank you for your patience!
This issue needs deeper investigation and Azure technical support team will be able to help with this. I would recommend you open an azure support case. If you don't have the ability to open a technical support ticket,
please let me know and I can help you further with this.
Roussel, Julien 0 Reputation points

2024-12-02T13:12:29.0733333+00:00

Im experiencing the same behavior too... everything was working fine until a few weeks ago.

Has Azure been able to fix this issue?
Srinud 3,400 Reputation points Microsoft Vendor

2024-12-03T11:09:58.6666667+00:00

Hi Roussel, Julien,

This issue needs deeper investigation and Azure technical support team will be able to help with this. I would recommend you open an azure support case. If you don't have the ability to open a technical support ticket, please let me know and I can help you further with this.
Thank you.

Share via

VMSS stopped suddenly doing AutoOSUpgrade

Your answer