Azure Arc Kubernetes - How do you enable AAD Authentication for a given cluster

Duncan House 0

How do you enable AAD Authentication for a given K8s cluster within Azure Arc. I want to manage the use of Azure RBAC for viewing the azure resource(s), namespaces, workloads, etc.

See image below

User's image

Duncan House 0

I have attempted to enable azure-rbac feature, using

az connectedk8s enable-features -n $clusterName -g $RG --features azure-rbac

# Output
PS C:\aksedge> az connectedk8s enable-features -n arc-k8s-vmem4ddjhdevbox -g aksedge-rg --features azure-rbac
D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\cryptography/hazmat/backends/openssl/backend.py:8: UserWarning: You are using cryptography on a 32-bit Python on a 64-bit Windows Operating System. Cryptography will be significantly faster if you switch to using a 64-bit Python.
This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
This operation might take a while...
Step: 2024-12-09T15-29-52Z: Validating custom access token
Step: 2024-12-09T15-29-53Z: Setting KubeConfig
Step: 2024-12-09T15-29-53Z: Checking Connectivity to Cluster
Step: 2024-12-09T15-29-53Z: Install Helm client if it does not exist
Step: 2024-12-09T15-29-53Z: Get namespace of release: azure-arc
Step: 2024-12-09T15-29-53Z: Getting HelmPackagePath from Arc DataPlane
Step: 2024-12-09T15-29-54Z: Determine Helmchart Export Path
Step: 2024-12-09T15-29-54Z: Pulling HelmChart: mcr.microsoft.com/azurearck8s/batch1/stable/v2/azure-arc-k8sagents, Version: 1.21.10
Please use the kubelogin version v0.0.32 or higher which has support for generating PoP token(s). This is needed by guard running in 'arc' authN mode.
"Successsfully enabled features: ['azure-rbac'] for the Connected Cluster arc-k8s-vmem4ddjhdevbox"

As you can see from the above, it suggests Azure RBAC has been enabled, but it doesn't appear to show in the Azure Portal.

Rahul Podila 805 Reputation points Microsoft Vendor

2024-12-12T09:07:39.5133333+00:00
Hi @Duncan House

Welcome to the Microsoft Q&A Platform! Thank you for asking your question here.

it can take a little time for the changes to show up in the Azure Portal. I’d recommend waiting about 5–10 minutes before refreshing and checking again.

To double-check that Azure RBAC is actually enabled, you can run this command:

az connectedk8s show -g <YourResourceGroup> -n <YourClusterName>

This will confirm if the azure-rbac feature is turned on for your cluster.

You’ll also need a tool called kubelogin to authenticate with Azure AD when using kubectl commands. If you don’t have it installed (or if it needs updating), you can install it with:

On macOS/Linux, use this:
brew install kubelogin

On Windows, download the latest version from this link.

Once installed, you can log in using Azure AD with:

kubelogin --azure-cli

Enabling Azure RBAC doesn’t automatically give users permission to access the Kubernetes resources. You’ll need to assign the correct roles to the users or groups who need access.

For example, if you want someone to view Kubernetes resources, you can assign them the "Azure Kubernetes Service RBAC Viewer" role like this:

az role assignment create --assignee <UserOrGroupObjectId> --role "Azure Kubernetes Service RBAC Viewer" --scope "/subscriptions/<YourSubscriptionId>/resourceGroups/<YourResourceGroup>/providers/Microsoft.Kubernetes/connectedClusters/<YourClusterName>"

After assigning roles, head to the Azure Portal > Azure Arc > Kubernetes Clusters. Select your cluster, and then go to the Access Control (IAM) tab. You should see the roles you’ve assigned to users there.

Finally, to make sure the permissions are working, you can run a quick test with this command:

kubectl auth can-i list pods --namespace <YourNamespace>

This checks if the user has permission to see the resources in the specified namespace.

Once you’ve followed these steps, Azure RBAC should be fully set up, and your users should be able to access Kubernetes resources based on their roles.

For reference, please review this documentation :-

https://learn.microsoft.com/en-us/azure/azure-arc/kubernetes/azure-rbac?tabs=kubernetes-latest#enable-azure-rbac-on-the-cluster

https://learn.microsoft.com/en-us/azure/azure-arc/kubernetes/overview

https://learn.microsoft.com/en-us/azure/azure-arc/kubernetes/azure-rbac?tabs=kubernetes-latest#enable-azure-rbac-on-the-cluster

If you have any further queries, do let us know.
Rahul Podila 805 Reputation points Microsoft Vendor

2024-12-13T01:29:13.6866667+00:00

Hi @Duncan House
If you had a chance to see my comment to your question. If it was helpful, please click "Upvote" on my post let us know Thank you...!

Duncan House 0

Thankyou first for your notes.

The AKS cluster is already connected (see below)

az connectedk8s show -n $clusterName -g $RG
{
  "aadProfile": {
    "adminGroupObjectIDs": null,
    "enableAzureRbac": null,
    "tenantId": null
  },

I cannot seem to run the below command, since it errors suggesting it is not connected. Any ideas how I resolve this?

az connectedk8s enable-features -n $clusterName -g $RG --features azure-rbac
This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
This operation might take a while...
Step: 2024-12-13T10-12-06Z: Validating custom access token
Step: 2024-12-13T10-12-07Z: Setting KubeConfig
Step: 2024-12-13T10-12-07Z: Checking Connectivity to Cluster
Step: 2024-12-13T10-12-07Z: Install Helm client if it does not exist
Step: 2024-12-13T10-12-07Z: Get namespace of release: azure-arc
The azure-arc release namespace couldn't be retrieved, which implies that the kubernetes cluster has not been onboarded to azure-arc.
Please run 'az connectedk8s connect -n <connected-cluster-name> -g <resource-group-name>' to onboard the cluster

The cluster is already connected.

Duncan House 0

Thankyou for providing a response.

I have been through the notes and I am having issues running the "enable-feature", since it is suggesting I am not connected.

Any ideas how I resolve this.

On the host I ran the following to connect to Azure Arc

PS C:\aksedge> Connect-AksEdgeArc -JsonConfigFilePath ./aksedge-config.json

[12/13/2024 09:22:05] *** No errors found in the connect Azure Arc configuration.

 - Got ClusterID after 0 retries.

[12/13/2024 09:22:06] AksEdge - Connecting cluster to Azure Arc

 - Checking Az PS module dependencies
 - Checking for NuGet
 - NuGet found
 - Az.Resources module with version 6.4.1 is found
 - Az.Accounts module with version 2.11.2 is found
 - Az.ConnectedKubernetes module with version 0.10.1 is found
 - Connecting to Azure Account
 - Verifying Azure Account connection ...
 - Verifying the Azure resource providers Microsoft.Kubernetes, Microsoft.KubernetesConfiguration, Microsoft.ExtendedLocation are registered
 - Resource provider Microsoft.Kubernetes is registered.
 - Resource provider Microsoft.KubernetesConfiguration is registered.
 - Resource provider Microsoft.ExtendedLocation is registered.
 - Checking whether cluster 'arc-k8s-vmem4ddjhdevbox' is connected to Azure Arc...
 - All checks succeeded. Connecting cluster to Azure Arc.
 - Populating tags for AKS-EE Cluster
 - Got ClusterID after 0 retries.
 - Connecting cluster to Azure...
 - Cluster reached connected status
OK

On my client I ran the following

az connectedk8s show -n $clusterName -g $RG
{
  "aadProfile": {
    "adminGroupObjectIDs": null,
    "enableAzureRbac": null,
    "tenantId": null
  },

az connectedk8s enable-features -n $clusterName -g $RG --features azure-rbac
This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
This operation might take a while...

Step: 2024-12-13T10-11-13Z: Validating custom access token
Step: 2024-12-13T10-11-13Z: Setting KubeConfig
Step: 2024-12-13T10-11-13Z: Checking Connectivity to Cluster
Step: 2024-12-13T10-11-13Z: Install Helm client if it does not exist
Step: 2024-12-13T10-11-13Z: Get namespace of release: azure-arc
The azure-arc release namespace couldn't be retrieved, which implies that the kubernetes cluster has not been onboarded to azure-arc.
Please run 'az connectedk8s connect -n <connected-cluster-name> -g <resource-group-name>' to onboard the cluster

Duncan House 0 Reputation points

2024-12-13T10:22:48.1933333+00:00

Can you enable Azure RBAC on creation, by updating the JSON file aksedge-config.json?

Duncan House 0

Hi @Rahul Podila

As you can see below, on another cluster I have managed to run the enable feature. Even though it says successful, the the feature is not enabled. Please advise.

duncanhouse@GGLV-L017552:/mnt/c/vscode$ az connectedk8s enable-features -n $clusterName -g $RG --features azure-rbac
This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
This operation might take a while...

Step: 2024-12-13T11-58-24Z: Validating custom access token
Step: 2024-12-13T11-58-24Z: Setting KubeConfig
Step: 2024-12-13T11-58-24Z: Checking Connectivity to Cluster
Step: 2024-12-13T11-58-24Z: Install Helm client if it does not exist
Step: 2024-12-13T11-58-24Z: Get namespace of release: azure-arc
Step: 2024-12-13T11-58-25Z: Getting HelmPackagePath from Arc DataPlane
Step: 2024-12-13T11-58-26Z: Determine Helmchart Export Path
Step: 2024-12-13T11-58-26Z: Pulling HelmChart: mcr.microsoft.com/azurearck8s/batch1/stable/v2/azure-arc-k8sagents, Version: 1.21.10
Please use the kubelogin version v0.0.32 or higher which has support for generating PoP token(s). This is needed by guard running in 'arc' authN mode.
"Successsfully enabled features: ['azure-rbac'] for the Connected Cluster duncanhouse-minikube"
duncanhouse@GGLV-L017552:/mnt/c/vscode$ az connectedk8s show -n $clusterName -g $RG --query aadProfile.{enableAzureRbac:enableAzureRbac} -o tsv
None
duncanhouse@GGLV-L017552:/mnt/c/vscode$ az connectedk8s show -n $clusterName -g $RG
{
  "aadProfile": {
    "adminGroupObjectIDs": null,
    "enableAzureRbac": null,
    "tenantId": null
  },

Rahul Podila 805 Reputation points Microsoft Vendor

2024-12-17T06:55:59.5233333+00:00
Hi @Duncan House

Thanks for providing all the details! It sounds like the cluster is having trouble fully registering with Azure Arc, even though the az connectedk8s enable-features command reports success. Let's go through a few steps to fix it:

Even though your cluster shows as connected, it seems there may be some issues with the connection. To fix this:

Disconnect the cluster from Azure Arc:

az connectedk8s delete -n <cluster-name> -g <resource-group>

Reconnect the cluster to Azure Arc:

az connectedk8s connect -n <cluster-name> -g <resource-group>

This will refresh the connection and make sure everything is registered correctly.

Once the cluster is reconnected, try enabling Azure RBAC again:

az connectedk8s enable-features -n <cluster-name> -g <resource-group> --features azure-rbac

This should properly enable the RBAC feature for your cluster.

Sometimes it takes a few minutes (5-10 minutes) for the change to take full effect. So run the command, wait a bit and then check again in the Azure Portal.

Go to Azure Arc > Kubernetes Clusters in the Azure Portal, and select your cluster. Then, check the Access Control (IAM) tab to see if Azure RBAC is showing up and if you can assign roles.

Since Azure RBAC uses Azure AD for authentication, you’ll need a tool called kubelogin. If you don’t have it installed yet, you can install it:

For macOS/Linux: Run brew install kubelogin
For Windows: Download it from GitHub.

Once installed, log in to Azure AD with:

kubelogin --azure-cli

To double-check if RBAC is actually enabled, run this command:

az connectedk8s show -n <cluster-name> -g <resource-group> --query aadProfile.enableAzureRbac

If it says true, then Azure RBAC is all set up!

You’ll need to assign roles to the users or groups that need access to Kubernetes resources. For example, to give someone view-only access, you can run:

az role assignment create --assignee <user-or-group> --role "Azure Kubernetes Service RBAC Viewer" --scope "/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Kubernetes/connectedClusters/<cluster-name>"

Once you’ve done all of this, you should be all good to go! If you still run into any issues or need more help, just let me know and we can dig a little deeper.

If you have any further queries, do let us know.
Duncan House 0 Reputation points

2024-12-17T10:59:06.91+00:00

@Rahul Podila Appreciate the support on this one.

No joy with your recommendations, I still get the same outcome as above.

Question: Does the enabling of RBAC require specific Azure RBAC permissions to enable this feature.
Rahul Podila 805 Reputation points Microsoft Vendor

2024-12-18T13:25:58.95+00:00

Hi @Duncan House To enable Role-Based Access Control (RBAC) in Azure, you'll need the correct permissions. Specifically, you need to be assigned either the Owner or User Access Administrator role.

If you don't already have one of these roles, you'll need to ask someone who has admin privileges in your organization to assign one to you. The role can be assigned at the subscription, resource group, or resource level, depending on what you need to manage.

Once you have the correct role, you'll be able to enable RBAC and manage access permissions for other users.

If you have any further queries, do let us know.
Duncan House 0 Reputation points

2024-12-18T18:58:12.6366667+00:00

@Rahul Podila I am the Subscription owner, so Azure RBAC for connected AKS clusters, isn't working. Can you demonstrate to me it works?
Rahul Podila 805 Reputation points Microsoft Vendor

2024-12-20T09:30:33.9866667+00:00
Hi
It seems that even though your cluster is connected, there might be issues with the registration. To ensure everything is set up correctly, let’s disconnect and reconnect your cluster:

az connectedk8s delete -n <YourClusterName> -g <YourResourceGroup> az connectedk8s connect -n <YourClusterName> -g <YourResourceGroup>

After reconnecting, try enabling Azure RBAC again:

az connectedk8s enable-features -n <YourClusterName> -g <YourResourceGroup> --features azure-rbac

Make sure to wait a few minutes after running this command before checking the Azure Portal again.

To confirm if RBAC is actually enabled, run the following command:

az connectedk8s show -n <YourClusterName> -g <YourResourceGroup> --query aadProfile.enableAzureRbac

If it returns true, then RBAC is successfully enabled.

Ensure you have kubelogin installed for authenticating with Azure AD when using kubectl. You can install it using:

For macOS/Linux:

brew install kubelogin

For Windows, download it from GitHub.

After confirming that RBAC is enabled, you’ll need to assign roles to users or groups who need access. For example, to give someone view-only access, use:

az role assignment create --assignee <UserOrGroupObjectId> --role "Azure Kubernetes Service RBAC Viewer" --scope "/subscriptions/<YourSubscriptionId>/resourceGroups/<YourResourceGroup>/providers/Microsoft.Kubernetes/connectedClusters/<YourClusterName>"

Since you’re the subscription owner, you should have the necessary permissions to enable RBAC. If you continue to face issues, please ensure there are no restrictions at the resource group or subscription level that might be affecting this.

If you have any further queries, do let us know.
Rahul Podila 805 Reputation points Microsoft Vendor

2024-12-23T01:13:16.29+00:00

Hi @Duncan House

We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion.
Duncan House 0 Reputation points

2024-12-23T07:55:57.58+00:00

Hi @Rahul Podila

I have tried several times to apply the delete followed by the connect and although the feature says it's enabled, and I assigned the RBAC role, it doesn't work.

I even asked co-pilot and checked that process and it still doesn't work.

https://learn.microsoft.com/en-us/azure/azure-arc/kubernetes/azure-rbac?tabs=kubernetes-latest

I am using minikube as the local provider for kubernetes on WSL. I have checked the OS architecture is amd64.

I am running out of ideas....
Duncan House 0 Reputation points

2024-12-23T08:20:47.84+00:00

Hi @Rahul Podila

I have tried the same process on both minikube and k3s and I cannot get Azure RBAC to show enabled.
Rahul Podila 805 Reputation points Microsoft Vendor

2024-12-26T02:55:09.2366667+00:00
Thanks for sharing all the details, and I totally understand your frustration. It seems like there might be some issues with the registration of your cluster in Azure Arc, even though it shows as "connected." Let’s go through a few steps to resolve it.

First, let's try disconnecting and reconnecting the cluster to refresh the connection. You can disconnect the cluster using the following command:

az connectedk8s delete -n <YourClusterName> -g <YourResourceGroup>

Then, reconnect it with this command:

az connectedk8s connect -n <YourClusterName> -g <YourResourceGroup>

Once the cluster is successfully reconnected, try enabling Azure RBAC again:

az connectedk8s enable-features -n <YourClusterName> -g <YourResourceGroup> --features azure-rbac

After running the command, wait for a few minutes for the changes to take effect. Then, check if RBAC is enabled by running this:

az connectedk8s show -n <YourClusterName> -g <YourResourceGroup> --query aadProfile.enableAzureRbac

Sure, here’s the answer without headings:

Hi @Duncan House,

Thanks for sharing all the details, and I totally understand your frustration. It seems like there might be some issues with the registration of your cluster in Azure Arc, even though it shows as "connected." Let’s go through a few steps to resolve it.

First, let's try disconnecting and reconnecting the cluster to refresh the connection. You can disconnect the cluster using the following command:

bash Copy code az connectedk8s delete -n <YourClusterName> -g <YourResourceGroup>

Then, reconnect it with this command:

bash Copy code az connectedk8s connect -n <YourClusterName> -g <YourResourceGroup>

Once the cluster is successfully reconnected, try enabling Azure RBAC again:

bash Copy code az connectedk8s enable-features -n <YourClusterName> -g <YourResourceGroup> --features azure-rbac

After running the command, wait for a few minutes for the changes to take effect. Then, check if RBAC is enabled by running this:

bash Copy code az connectedk8s show -n <YourClusterName> -g <YourResourceGroup> --query aadProfile.enableAzureRbac

If it returns true, Azure RBAC is enabled and you’re good to go. If it still shows null, we might need to troubleshoot further.

Once RBAC is enabled, you’ll need to assign roles to the users or groups who need access. For example, to give someone view-only access, you can assign the "Azure Kubernetes Service RBAC Viewer" role:

az role assignment create --assignee <UserOrGroupObjectId> --role "Azure Kubernetes Service RBAC Viewer" --scope "/subscriptions/<YourSubscriptionId>/resourceGroups/<YourResourceGroup>/providers/Microsoft.Kubernetes/connectedClusters/<YourClusterName>"

Since Azure RBAC uses Azure Active Directory for authentication, you’ll also need to install kubelogin for Azure AD authentication. If you don’t have it installed yet, you can do so with:

brew install kubelogin

Once kubelogin is installed, log in to Azure AD using:

kubelogin --azure-cli

Finally, to check if everything is working as expected, you can test the permissions with this command:

kubectl auth can-i get pods --namespace <YourNamespace>

This will confirm whether the user has access to the resources in the specified namespace.

A couple of things to keep in mind: Since you're using Minikube and k3s, these are local Kubernetes environments, and sometimes they don’t fully support all features of Azure Arc or Azure RBAC. If possible, I would recommend testing this process with Azure Kubernetes Service (AKS) to make sure everything works as expected. Also, after running the commands, allow a few minutes for changes to show up in the Azure Portal.
If any references please go through this link:

https://learn.microsoft.com/en-us/azure/azure-arc/kubernetes/overview

https://learn.microsoft.com/en-us/azure/azure-arc/kubernetes/azure-rbac?tabs=kubernetes-latest#enable-azure-rbac-on-the-cluster

If you have any further queries, do let us know.

Share via

Azure Arc Kubernetes - How do you enable AAD Authentication for a given cluster

Your answer