Resource 'aksManagedAutoUpgradeSchedule' was disallowed by policy. What is the resource type?

Galati, Domenic-GALATD4 0

I need to whitelist a resource type in Azure to enable AutoUpgrade scheduler. Otherwise policy disallows. But I don't know which resource type to whitelist.

Pavan Minukuri 520 Reputation points Microsoft Vendor

2024-11-07T00:42:42.88+00:00
Hi @Galati, Domenic-GALATD4
Welcome to the Microsoft Q&A Platform! Thank you for asking your question here.

The resource type 'aksManagedAutoUpgradeSchedule' refers to the Azure Kubernetes Service (AKS) managed cluster auto-upgrade schedule resource. This resource type is used to configure the auto-upgrade settings for an AKS managed cluster.

The AKS managed cluster auto-upgrade schedule resource allows you to specify the maintenance window and upgrade settings for your AKS managed cluster. You can use this resource to schedule automatic upgrades for your cluster during a specific maintenance window, and configure the upgrade settings such as the upgrade channel, max surge, and max unavailable nodes.

The error message "Resource 'aksManagedAutoUpgradeSchedule' was disallowed by policy" indicates that there is an Azure policy in place that disallows the creation of this resource type. This policy might be preventing you from creating or modifying the auto-upgrade schedule for your AKS managed cluster. You can check the Azure policy settings to see if there is a policy that is blocking the creation of this resource type.
To enable the AKS managed cluster auto-upgrade schedule resource, you need to whitelist the 'Microsoft.ContainerService/managedClusters/autoUpgradeProfiles' resource type in Azure Policy.

Please follow the below steps:

Go to the Azure Policy blade in the Azure portal.

Click on the 'Definitions' tab and search for the policy that is blocking the creation of the AKS managed cluster auto-upgrade schedule resource.

Click on the policy to open the policy details page.

Click on the 'Edit' button to edit the policy.

In the 'Policy rule' section, click on the 'Add' button to add a new condition.

In the 'Add condition' dialog box, select 'Resource types' as the condition type.

In the 'Resource types' field, enter 'Microsoft.ContainerService/managedClusters/autoUpgradeProfiles' as the resource type.

Click on the 'Add' button to add the condition.

Click on the 'Review + create' button to review and create the updated policy.

Click on the 'Create' button to create the updated policy.

Please refer attached link for better understanding: https://learn.microsoft.com/en-us/azure/governance/policy/overview https://learn.microsoft.com/en-us/azure/governance/policy/concepts/policy-for-kubernetes

If you required anything please let me, If the Commnet is helpful, please click "Accept Comment ".
Pavan Minukuri 520 Reputation points Microsoft Vendor

2024-11-07T17:36:17.2+00:00

Hi @Galati, Domenic-GALATD4
Just want to check is that issue resolved or else if you need any help, please let us know we will always help as you needed.! if you had chance please review once my previous comment. Thank you.!
Galati, Domenic-GALATD4 0 Reputation points

2024-11-07T19:14:09.4366667+00:00

Thanks Pavan, for providing the resource type for policy whitelisting. We have yet to apply the policy exception and unfortunately, because the policy exception process is time consuming, we have to be confident before we request it. Some of our colleagues and also Microsoft support are suggesting these resource type are required:
Microsoft.ContainerService/managedClusters//maintenanceConfigurations/aksManagedAutoUpgradeScheduleMicrosoft.ContainerService/managedClusters//maintenanceConfigurations/aksManagedNodeOSUpgradeSchedule

You are suggesting:

Microsoft.ContainerService/managedClusters/autoUpgradeProfiles

Do I have a case to whitelist all of these?
Pavan Minukuri 520 Reputation points Microsoft Vendor

2024-11-08T19:42:20.1433333+00:00

Hi @Galati, Domenic-GALATD4
We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion.
Thnank you.
Galati, Domenic-GALATD4 0 Reputation points

2024-11-08T20:33:42.2566667+00:00

Pavan, once again, thank you for clarifying. I am going to recommend enabling policy exceptions for all 3 resource types. It only makes sense if you are going to schedule maintenance (which we intend to do), you will need to enable the autoUpgrade feature first.

I can mark this as an accepted answer or should I wait until we apply the policy changes?
Pavan Minukuri 520 Reputation points Microsoft Vendor

2024-11-08T23:25:00.2666667+00:00

Hi @Galati, Domenic-GALATD4
It's generally best to wait until you've applied the policy changes after you can accept answer.
Thank You!
Pavan Minukuri 520 Reputation points Microsoft Vendor

2024-11-12T17:13:52.86+00:00

Hi @Galati, Domenic-GALATD4
Just want to check is that issue resolved or else if you need any help, please let us know we will always help as you needed.! if you had chance please review once my previous comment. Thank you.!

Accepted answer

Pavan Minukuri 520 Reputation points Microsoft Vendor

2024-11-07T21:30:07.3066667+00:00

Thanks for replying Galati, Domenic-GALATD4

The resource types that you need to whitelist in Azure Policy depend on the specific features and settings that you want to enable for your AKS managed cluster.

The resource type 'Microsoft.ContainerService/managedClusters/autoUpgradeProfiles' is used to configure the auto-upgrade settings for an AKS managed cluster. This resource type is required if you want to enable the AKS managed cluster auto-upgrade feature.

The resource types 'Microsoft.ContainerService/managedClusters/maintenanceConfigurations/aksManagedAutoUpgradeSchedule' and 'Microsoft.ContainerService/managedClusters/maintenanceConfigurations/aksManagedNodeOSUpgradeSchedule' are used to configure the maintenance window and upgrade settings for an AKS managed cluster. These resource types are required if you want to enable the AKS managed cluster maintenance feature.

If you want to enable both the auto-upgrade and maintenance features for your AKS managed cluster, you need to whitelist all three resource types. If you only want to enable the auto-upgrade feature, you only need to whitelist the 'Microsoft.ContainerService/managedClusters/autoUpgradeProfiles' resource type.

Finally, the resource types that you need to whitelist in Azure Policy depend on the specific features and settings that you want to enable for your AKS managed cluster. You can whitelist all three resource types if you want to enable both the auto-upgrade and maintenance features, or you can whitelist only the 'Microsoft.ContainerService/managedClusters/autoUpgradeProfiles' resource type if you only want to enable the auto-upgrade feature.

If you required anything, please let me.

Thank you!
Please sign in to rate this answer.
Pavan Minukuri 520 Reputation points Microsoft Vendor

2024-11-13T16:28:57.47+00:00

Hi @Galati, Domenic-GALATD4
Just want to check is that issue resolved or else if you need any help, please let us know we will always help as you needed.! if you had chance please review once my previous comment. Thank you.!

Pavan Minukuri 520 Reputation points Microsoft Vendor

2024-11-14T16:23:54.9033333+00:00

Hi @Galati, Domenic-GALATD4
We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion. Thank you...!

Galati, Domenic-GALATD4 0 Reputation points

2024-11-15T14:11:59.2866667+00:00

Hello Pavan
After some discussion and review of past failed compliance logs we determined that only the following resource type needed to be whitelisted.
Microsoft.ContainerService/managedClusters/maintenanceConfigurations
We are not done testing, but so far this allows us to enable and schedule node OS upgrades and AKS cluster upgrades without error. We are still waiting to see the actual execution of the updates.
Admittedly I don't understand the tree structure of these resource types. Does this mean that all resources types with the root above are whitelisted?

Pavan Minukuri 520 Reputation points Microsoft Vendor

2024-11-15T17:51:24.2833333+00:00

Hi @Galati, Domenic-GALATD4
It's great to hear that you were able to determine the specific resource type that needed to be whitelisted in order to enable and schedule node OS upgrades and AKS cluster upgrades without error. In response to your question, whitelisting a specific resource type such as "Microsoft.ContainerService/managedClusters/maintenanceConfigurations" will only whitelist that specific resource type and not any other resource types with a different root.

The Azure resource type hierarchy is organized in a tree structure, with each resource type having a unique identifier that includes the resource provider, resource type, and optionally, the resource name. The root of the tree structure is the Azure resource provider, which is responsible for managing a specific set of resource types. For example, the "Microsoft.ContainerService" resource provider is responsible for managing resources related to Azure Kubernetes Service (AKS).

When you whitelist a specific resource type, you are allowing that resource type to be created or modified without triggering any policy violations. However, other resource types with a different root will still be subject to policy enforcement. If you need to whitelist additional resource types, you will need to identify them and add them to your policy whitelist.

I hope this helps! Let me know if you have any other questions.

Galati, Domenic-GALATD4 0 Reputation points

2024-11-15T18:49:51.96+00:00

Oh, ok i was including the resource names as part of the resource type.

In other words:

This resource type:

Microsoft.ContainerService/managedClusters/maintenanceConfigurations

will cover both resources below:

aksManagedNodeOSUpgradeSchedule

aksManagedAutoUpgradeSchedule

without the need to specify the resource

Pavan Minukuri 520 Reputation points Microsoft Vendor

2024-11-15T19:09:04.9266667+00:00

Yes, that's correct! The resource type "Microsoft.ContainerService/managedClusters/maintenanceConfigurations" covers both "aksManagedNodeOSUpgradeSchedule" and "aksManagedAutoUpgradeSchedule" resources, so you don't need to specify the resource name in your policy. When you whitelist the "Microsoft.ContainerService/managedClusters/maintenanceConfigurations" resource type, it will allow any resources of that type to be created or modified without triggering any policy violations.
Thank You...!

Pavan Minukuri 520 Reputation points Microsoft Vendor

2024-11-18T16:51:43.5366667+00:00

Hi @Galati, Domenic-GALATD4
We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion. Thank you

Pavan Minukuri 520 Reputation points Microsoft Vendor

2024-11-19T19:20:02.42+00:00

Hi @Galati, Domenic-GALATD4
Just want to check is that issue resolved or else if you need any help, please let us know we will always help as you needed.! if you had chance please review once my previous comment. Thank you

Galati, Domenic-GALATD4 0 Reputation points

2024-11-20T18:52:52.9066667+00:00

Hello Pavan. We have indeed created the policy exceptions for microsoft.ContainerService/managedClusters/maintenanceConfigurations and we are now able to schedule upgrade without issue! Is there any way to determine that any action had taken place in the patching process? Currently I am just monitoring the node patch version az aks nodepool list --resource-group $RG --cluster-name $CL --query '[].{Name:name, Version:orchestratorVersion, OS:osType, KubernetesVersion:kubernetesVersion, NodeImageVersion:nodeImageVersion}

Galati, Domenic-GALATD4 0 Reputation points

2024-11-20T19:58:58+00:00

Node patch did upgrade. Kubernetes patch level was already up to date.

Pavan Minukuri 520 Reputation points Microsoft Vendor

2024-11-20T20:16:14.8866667+00:00

Thanks for replying Galati, Domenic-GALATD4
You can monitor the patching process for your AKS nodes by using Azure Monitor.

To do this, you can follow these steps:

Enable AKS monitoring in Azure Monitor. You can do this by going to the "Monitoring" blade for your AKS cluster and selecting "Enable monitoring". This will create a Log Analytics workspace and configure your AKS cluster to send logs to the workspace.

Create a Log Analytics query to monitor the patching process. You can use the "ContainerNodeInventory" table in Log Analytics to monitor the patch version of your AKS nodes. For example, you can use the following query to get the patch version of all nodes in your AKS cluster:

ContainerNodeInventory |

This query will return a list of all nodes in your AKS cluster, along with their patch version and other metadata.

Create an Azure Monitor alert to notify you of any patching issues. You can create an alert in Azure Monitor to notify you if any nodes in your AKS cluster are not up-to-date with the latest patches. For example, you can create an alert that triggers if any nodes have a patch version that is more than 7 days old.

If the node patch was upgraded successfully, you can verify the patch level of the nodes by running the following command:

az aks nodepool list

This command will return a list of all node pools in your AKS cluster, along with their patch level and other metadata.

If the Kubernetes patch level was already up to date, you can verify the version of Kubernetes by running the following command:

az aks show --resource-

This command will return the version of Kubernetes that is currently running in your AKS cluster.

I hope this helps! Let me know if you have any further questions.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

1 additional answer

Galati, Domenic-GALATD4 0 Reputation points

2024-11-20T21:46:40.66+00:00

thank you Pavan for your in depth responses!
Please sign in to rate this answer.

0 comments No comments
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Resource 'aksManagedAutoUpgradeSchedule' was disallowed by policy. What is the resource type?

1 additional answer

Your answer