Azure Arc Kubernetes - How do you enable AAD Authentication for a given cluster

Duncan House 0

How do you enable AAD Authentication for a given K8s cluster within Azure Arc. I want to manage the use of Azure RBAC for viewing the azure resource(s), namespaces, workloads, etc.

See image below

User's image

Duncan House 0

I have attempted to enable azure-rbac feature, using

az connectedk8s enable-features -n $clusterName -g $RG --features azure-rbac

# Output
PS C:\aksedge> az connectedk8s enable-features -n arc-k8s-vmem4ddjhdevbox -g aksedge-rg --features azure-rbac
D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\cryptography/hazmat/backends/openssl/backend.py:8: UserWarning: You are using cryptography on a 32-bit Python on a 64-bit Windows Operating System. Cryptography will be significantly faster if you switch to using a 64-bit Python.
This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
This operation might take a while...
Step: 2024-12-09T15-29-52Z: Validating custom access token
Step: 2024-12-09T15-29-53Z: Setting KubeConfig
Step: 2024-12-09T15-29-53Z: Checking Connectivity to Cluster
Step: 2024-12-09T15-29-53Z: Install Helm client if it does not exist
Step: 2024-12-09T15-29-53Z: Get namespace of release: azure-arc
Step: 2024-12-09T15-29-53Z: Getting HelmPackagePath from Arc DataPlane
Step: 2024-12-09T15-29-54Z: Determine Helmchart Export Path
Step: 2024-12-09T15-29-54Z: Pulling HelmChart: mcr.microsoft.com/azurearck8s/batch1/stable/v2/azure-arc-k8sagents, Version: 1.21.10
Please use the kubelogin version v0.0.32 or higher which has support for generating PoP token(s). This is needed by guard running in 'arc' authN mode.
"Successsfully enabled features: ['azure-rbac'] for the Connected Cluster arc-k8s-vmem4ddjhdevbox"

As you can see from the above, it suggests Azure RBAC has been enabled, but it doesn't appear to show in the Azure Portal.

Rahul Podila 800 Reputation points Microsoft Vendor

2024-12-12T09:07:39.5133333+00:00
Hi @Duncan House

Welcome to the Microsoft Q&A Platform! Thank you for asking your question here.

it can take a little time for the changes to show up in the Azure Portal. I’d recommend waiting about 5–10 minutes before refreshing and checking again.

To double-check that Azure RBAC is actually enabled, you can run this command:

az connectedk8s show -g <YourResourceGroup> -n <YourClusterName>

This will confirm if the azure-rbac feature is turned on for your cluster.

You’ll also need a tool called kubelogin to authenticate with Azure AD when using kubectl commands. If you don’t have it installed (or if it needs updating), you can install it with:

On macOS/Linux, use this:
brew install kubelogin

On Windows, download the latest version from this link.

Once installed, you can log in using Azure AD with:

kubelogin --azure-cli

Enabling Azure RBAC doesn’t automatically give users permission to access the Kubernetes resources. You’ll need to assign the correct roles to the users or groups who need access.

For example, if you want someone to view Kubernetes resources, you can assign them the "Azure Kubernetes Service RBAC Viewer" role like this:

az role assignment create --assignee <UserOrGroupObjectId> --role "Azure Kubernetes Service RBAC Viewer" --scope "/subscriptions/<YourSubscriptionId>/resourceGroups/<YourResourceGroup>/providers/Microsoft.Kubernetes/connectedClusters/<YourClusterName>"

After assigning roles, head to the Azure Portal > Azure Arc > Kubernetes Clusters. Select your cluster, and then go to the Access Control (IAM) tab. You should see the roles you’ve assigned to users there.

Finally, to make sure the permissions are working, you can run a quick test with this command:

kubectl auth can-i list pods --namespace <YourNamespace>

This checks if the user has permission to see the resources in the specified namespace.

Once you’ve followed these steps, Azure RBAC should be fully set up, and your users should be able to access Kubernetes resources based on their roles.

For reference, please review this documentation :-

https://learn.microsoft.com/en-us/azure/azure-arc/kubernetes/azure-rbac?tabs=kubernetes-latest#enable-azure-rbac-on-the-cluster

https://learn.microsoft.com/en-us/azure/azure-arc/kubernetes/overview

https://learn.microsoft.com/en-us/azure/azure-arc/kubernetes/azure-rbac?tabs=kubernetes-latest#enable-azure-rbac-on-the-cluster

If you have any further queries, do let us know.
Rahul Podila 800 Reputation points Microsoft Vendor

2024-12-13T01:29:13.6866667+00:00

Hi @Duncan House
If you had a chance to see my comment to your question. If it was helpful, please click "Upvote" on my post let us know Thank you...!

Duncan House 0

Thankyou first for your notes.

The AKS cluster is already connected (see below)

az connectedk8s show -n $clusterName -g $RG
{
  "aadProfile": {
    "adminGroupObjectIDs": null,
    "enableAzureRbac": null,
    "tenantId": null
  },

I cannot seem to run the below command, since it errors suggesting it is not connected. Any ideas how I resolve this?

az connectedk8s enable-features -n $clusterName -g $RG --features azure-rbac
This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
This operation might take a while...
Step: 2024-12-13T10-12-06Z: Validating custom access token
Step: 2024-12-13T10-12-07Z: Setting KubeConfig
Step: 2024-12-13T10-12-07Z: Checking Connectivity to Cluster
Step: 2024-12-13T10-12-07Z: Install Helm client if it does not exist
Step: 2024-12-13T10-12-07Z: Get namespace of release: azure-arc
The azure-arc release namespace couldn't be retrieved, which implies that the kubernetes cluster has not been onboarded to azure-arc.
Please run 'az connectedk8s connect -n <connected-cluster-name> -g <resource-group-name>' to onboard the cluster

The cluster is already connected.

Duncan House 0

Thankyou for providing a response.

I have been through the notes and I am having issues running the "enable-feature", since it is suggesting I am not connected.

Any ideas how I resolve this.

On the host I ran the following to connect to Azure Arc

PS C:\aksedge> Connect-AksEdgeArc -JsonConfigFilePath ./aksedge-config.json

[12/13/2024 09:22:05] *** No errors found in the connect Azure Arc configuration.

 - Got ClusterID after 0 retries.

[12/13/2024 09:22:06] AksEdge - Connecting cluster to Azure Arc

 - Checking Az PS module dependencies
 - Checking for NuGet
 - NuGet found
 - Az.Resources module with version 6.4.1 is found
 - Az.Accounts module with version 2.11.2 is found
 - Az.ConnectedKubernetes module with version 0.10.1 is found
 - Connecting to Azure Account
 - Verifying Azure Account connection ...
 - Verifying the Azure resource providers Microsoft.Kubernetes, Microsoft.KubernetesConfiguration, Microsoft.ExtendedLocation are registered
 - Resource provider Microsoft.Kubernetes is registered.
 - Resource provider Microsoft.KubernetesConfiguration is registered.
 - Resource provider Microsoft.ExtendedLocation is registered.
 - Checking whether cluster 'arc-k8s-vmem4ddjhdevbox' is connected to Azure Arc...
 - All checks succeeded. Connecting cluster to Azure Arc.
 - Populating tags for AKS-EE Cluster
 - Got ClusterID after 0 retries.
 - Connecting cluster to Azure...
 - Cluster reached connected status
OK

On my client I ran the following

az connectedk8s show -n $clusterName -g $RG
{
  "aadProfile": {
    "adminGroupObjectIDs": null,
    "enableAzureRbac": null,
    "tenantId": null
  },

az connectedk8s enable-features -n $clusterName -g $RG --features azure-rbac
This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
This operation might take a while...

Step: 2024-12-13T10-11-13Z: Validating custom access token
Step: 2024-12-13T10-11-13Z: Setting KubeConfig
Step: 2024-12-13T10-11-13Z: Checking Connectivity to Cluster
Step: 2024-12-13T10-11-13Z: Install Helm client if it does not exist
Step: 2024-12-13T10-11-13Z: Get namespace of release: azure-arc
The azure-arc release namespace couldn't be retrieved, which implies that the kubernetes cluster has not been onboarded to azure-arc.
Please run 'az connectedk8s connect -n <connected-cluster-name> -g <resource-group-name>' to onboard the cluster

Duncan House 0 Reputation points

2024-12-13T10:22:48.1933333+00:00

Can you enable Azure RBAC on creation, by updating the JSON file aksedge-config.json?

Duncan House 0

Hi @Rahul Podila

As you can see below, on another cluster I have managed to run the enable feature. Even though it says successful, the the feature is not enabled. Please advise.

duncanhouse@GGLV-L017552:/mnt/c/vscode$ az connectedk8s enable-features -n $clusterName -g $RG --features azure-rbac
This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
This operation might take a while...

Step: 2024-12-13T11-58-24Z: Validating custom access token
Step: 2024-12-13T11-58-24Z: Setting KubeConfig
Step: 2024-12-13T11-58-24Z: Checking Connectivity to Cluster
Step: 2024-12-13T11-58-24Z: Install Helm client if it does not exist
Step: 2024-12-13T11-58-24Z: Get namespace of release: azure-arc
Step: 2024-12-13T11-58-25Z: Getting HelmPackagePath from Arc DataPlane
Step: 2024-12-13T11-58-26Z: Determine Helmchart Export Path
Step: 2024-12-13T11-58-26Z: Pulling HelmChart: mcr.microsoft.com/azurearck8s/batch1/stable/v2/azure-arc-k8sagents, Version: 1.21.10
Please use the kubelogin version v0.0.32 or higher which has support for generating PoP token(s). This is needed by guard running in 'arc' authN mode.
"Successsfully enabled features: ['azure-rbac'] for the Connected Cluster duncanhouse-minikube"
duncanhouse@GGLV-L017552:/mnt/c/vscode$ az connectedk8s show -n $clusterName -g $RG --query aadProfile.{enableAzureRbac:enableAzureRbac} -o tsv
None
duncanhouse@GGLV-L017552:/mnt/c/vscode$ az connectedk8s show -n $clusterName -g $RG
{
  "aadProfile": {
    "adminGroupObjectIDs": null,
    "enableAzureRbac": null,
    "tenantId": null
  },

Rahul Podila 800 Reputation points Microsoft Vendor

2024-12-17T06:55:59.5233333+00:00
Hi @Duncan House

Thanks for providing all the details! It sounds like the cluster is having trouble fully registering with Azure Arc, even though the az connectedk8s enable-features command reports success. Let's go through a few steps to fix it:

Even though your cluster shows as connected, it seems there may be some issues with the connection. To fix this:

Disconnect the cluster from Azure Arc:

az connectedk8s delete -n <cluster-name> -g <resource-group>

Reconnect the cluster to Azure Arc:

az connectedk8s connect -n <cluster-name> -g <resource-group>

This will refresh the connection and make sure everything is registered correctly.

Once the cluster is reconnected, try enabling Azure RBAC again:

az connectedk8s enable-features -n <cluster-name> -g <resource-group> --features azure-rbac

This should properly enable the RBAC feature for your cluster.

Sometimes it takes a few minutes (5-10 minutes) for the change to take full effect. So run the command, wait a bit and then check again in the Azure Portal.

Go to Azure Arc > Kubernetes Clusters in the Azure Portal, and select your cluster. Then, check the Access Control (IAM) tab to see if Azure RBAC is showing up and if you can assign roles.

Since Azure RBAC uses Azure AD for authentication, you’ll need a tool called kubelogin. If you don’t have it installed yet, you can install it:

For macOS/Linux: Run brew install kubelogin
For Windows: Download it from GitHub.

Once installed, log in to Azure AD with:

kubelogin --azure-cli

To double-check if RBAC is actually enabled, run this command:

az connectedk8s show -n <cluster-name> -g <resource-group> --query aadProfile.enableAzureRbac

If it says true, then Azure RBAC is all set up!

You’ll need to assign roles to the users or groups that need access to Kubernetes resources. For example, to give someone view-only access, you can run:

az role assignment create --assignee <user-or-group> --role "Azure Kubernetes Service RBAC Viewer" --scope "/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Kubernetes/connectedClusters/<cluster-name>"

Once you’ve done all of this, you should be all good to go! If you still run into any issues or need more help, just let me know and we can dig a little deeper.

If you have any further queries, do let us know.
Duncan House 0 Reputation points

2024-12-17T10:59:06.91+00:00

@Rahul Podila Appreciate the support on this one.

No joy with your recommendations, I still get the same outcome as above.

Question: Does the enabling of RBAC require specific Azure RBAC permissions to enable this feature.
Rahul Podila 800 Reputation points Microsoft Vendor

2024-12-18T13:25:58.95+00:00

Hi @Duncan House To enable Role-Based Access Control (RBAC) in Azure, you'll need the correct permissions. Specifically, you need to be assigned either the Owner or User Access Administrator role.

If you don't already have one of these roles, you'll need to ask someone who has admin privileges in your organization to assign one to you. The role can be assigned at the subscription, resource group, or resource level, depending on what you need to manage.

Once you have the correct role, you'll be able to enable RBAC and manage access permissions for other users.

If you have any further queries, do let us know.
Duncan House 0 Reputation points

2024-12-18T18:58:12.6366667+00:00

@Rahul Podila I am the Subscription owner, so Azure RBAC for connected AKS clusters, isn't working. Can you demonstrate to me it works?
Rahul Podila 800 Reputation points Microsoft Vendor

2024-12-20T09:30:33.9866667+00:00
Hi
It seems that even though your cluster is connected, there might be issues with the registration. To ensure everything is set up correctly, let’s disconnect and reconnect your cluster:

az connectedk8s delete -n <YourClusterName> -g <YourResourceGroup> az connectedk8s connect -n <YourClusterName> -g <YourResourceGroup>

After reconnecting, try enabling Azure RBAC again:

az connectedk8s enable-features -n <YourClusterName> -g <YourResourceGroup> --features azure-rbac

Make sure to wait a few minutes after running this command before checking the Azure Portal again.

To confirm if RBAC is actually enabled, run the following command:

az connectedk8s show -n <YourClusterName> -g <YourResourceGroup> --query aadProfile.enableAzureRbac

If it returns true, then RBAC is successfully enabled.

Ensure you have kubelogin installed for authenticating with Azure AD when using kubectl. You can install it using:

For macOS/Linux:

brew install kubelogin

For Windows, download it from GitHub.

After confirming that RBAC is enabled, you’ll need to assign roles to users or groups who need access. For example, to give someone view-only access, use:

az role assignment create --assignee <UserOrGroupObjectId> --role "Azure Kubernetes Service RBAC Viewer" --scope "/subscriptions/<YourSubscriptionId>/resourceGroups/<YourResourceGroup>/providers/Microsoft.Kubernetes/connectedClusters/<YourClusterName>"

Since you’re the subscription owner, you should have the necessary permissions to enable RBAC. If you continue to face issues, please ensure there are no restrictions at the resource group or subscription level that might be affecting this.

If you have any further queries, do let us know.

Share via

Azure Arc Kubernetes - How do you enable AAD Authentication for a given cluster

Your answer