Deploy and configure Workload Identity on an AKS enabled by Azure Arc cluster (preview)

Article
11/25/2024

Applies to: Azure Local, version 23H2

Workload identity federation allows you to configure a user-assigned managed identity or app registration in Microsoft Entra ID to trust tokens from an external identity provider (IdP), such as Kubernetes, enabling access to resources protected by Microsoft Entra, like Azure Key Vault or Azure Blob storage.

Azure Kubernetes Service (AKS) enabled by Azure Arc is a managed Kubernetes service that lets you easily deploy workload identity enabled Kubernetes clusters. This article describes how to perform the following tasks:

Create an AKS Arc cluster with workload identity enabled (preview).
Create a Kubernetes service account and bind it to the Azure Managed Identity.
Create a federated credential on the managed identity to trust the OIDC issuer.
Deploy your application.
Example: Grant a pod in the cluster access to secrets in an Azure key vault.

For a conceptual overview of Workload identity federation, see Workload identity federation in Azure Arc-enabled Kubernetes (preview).

Important

These preview features are available on a self-service, opt-in basis. Previews are provided "as is" and "as available," and they're excluded from the service-level agreements and limited warranty. Azure Kubernetes Service, enabled by Azure Arc previews are partially covered by customer support on a best-effort basis.

Note

In public preview, AKS on Azure Local, version 23H2 supports enabling workload identity during AKS cluster creation. However, enabling workload identity after cluster creation or disabling it afterward is currently unsupported.

Prerequisites

Before you deploy a Kubernetes cluster with Azure Arc enabled, you must have the following prerequisites:

If you don't have an Azure subscription, create a free account before you begin.
This article requires version 1.4.23 or later of the Azure CLI. If you're using Azure Cloud Shell, the latest version is already installed.

Export environment variables

To help simplify the steps to configure the identities required, the following commands define environment variables that are referenced in the examples in this article. Replace the following values with your own:

$AZSubscriptionID = "00000000-0000-0000-0000-000000000000" 
$Location = "westeurope" 
$resource_group_name = "myResourceGroup" 

$aks_cluster_name = "myAKSCluster" 

$SERVICE_ACCOUNT_NAMESPACE = "default" 
$SERVICE_ACCOUNT_NAME = "workload-identity-sa" 

$FedIdCredentialName = "myFedIdentity" 
$MSIName = "myIdentity" 

# To access key vault secrets from a pod in the cluster, include these variables:
$KVName = "KV-workload-id" 
$KVSecretName= "KV-secret"

Set the active subscription

First, set your subscription as the current active subscription. Run the az account set command with your subscription ID:

az login  
az account set -s $AZSubscriptionID

Create a resource group

An Azure resource group is a logical group in which Azure resources are deployed and managed. When you create a resource group, you're prompted to specify a location. This location is the storage location of your resource group metadata and where your resources run in Azure if you don't specify another region during resource creation.

To create a resource group, run the az group create command:

az group create --name $resource_group_name --location $Location

The following example output shows the successful creation of a resource group:

{ 
  "id": "/subscriptions/<guid>/resourceGroups/myResourceGroup", 
  "location": "westeurope", 
  "managedBy": null, 
  "name": "$resource_group_name", 
  "properties": { 
    "provisioningState": "Succeeded" 
  }, 
  "tags": null 
}

Step 1: Create an AKS Arc cluster with workload identity enabled

To create an AKS Arc cluster, you need both the $customlocation_ID and $logicnet_Id values.

$customlocation_ID: The Azure Resource Manager ID of the custom location. The custom location is configured during the Azure Local, version 23H2 cluster deployment. Your infrastructure admin should give you the Resource Manager ID of the custom location. You can also get the Resource Manager ID using $customlocation_ID = $(az customlocation show --name "<your-custom-location-name>" --resource-group $resource_group_name --query "id" -o tsv), if the infrastructure admin provides a custom location name and resource group name.
$logicnet_Id: The Azure Resource Manager ID of the Azure Local logical network created following these steps. Your infrastructure admin should give you the Resource Manager ID of the logical network. You can also get the Resource Manager ID using $logicnet_Id = $(az stack-hci-vm network lnet show --name "<your-lnet-name>" --resource-group $resource_group_name --query "id" -o tsv), if the infrastructure admin provides a logical network name and resource group name.

Run the az aksarc create command with the --enable-oidc-issuer --enable-workload-identity parameter. Provide your entra-admin-group-object-ids and ensure you're a member of the Microsoft Entra ID admin group for proxy mode access:

az aksarc create  
-n $aks_cluster_name -g $resource_group_name  
--custom-location $customlocation_ID --vnet-ids $logicnet_Id  
--aad-admin-group-object-ids <entra-admin-group-object-ids> 
--generate-ssh-keys  
--enable-oidc-issuer --enable-workload-identity

After a few minutes, the command completes and returns JSON-formatted information about the cluster.

It might take some time for the workload identity extension to be deployed after successfully creating a provisioned cluster. Use the following command to check the workload identity extension status:

az connectedk8s show -n $aks_cluster_name -g $resource_group_name

# agentState = "Succeeded" 
"agentPublicKeyCertificate": "", 
  "agentVersion": "1.21.10", 
  "arcAgentProfile": { 
    "agentAutoUpgrade": "Enabled", 
    "agentErrors": [], 
    "agentState": "Succeeded", 
    "desiredAgentVersion": "", 
    "systemComponents": null 

# oidcIssuerProfile "enabled": true and "issuerUrl" present 

"oidcIssuerProfile": { 
    "enabled": true, 
    "issuerUrl": "https://oidcdiscovery-{location}-endpoint-1111111111111111.000.azurefd.net/00000000-0000-0000-0000-000000000000/11111111-1111-1111-1111-111111111111/"}

In the Azure portal, you can view the wiextension extension under the Properties section of your Kubernetes cluster.

Important

As part of the security enhancement for AKS Arc clusters, workload identity enablement triggers two changes. First, the Kubernetes service account signing key automatically rotates every 45 days and remains valid for 90 days. Second, the --service-account-extend-token-expiration flag is disabled, reducing token validity from one year to a maximum of 24 hours.

Save the OIDC issuer URL to an environment variable

Once AKS cluster is created successfully, you can get the OIDC issuer URL and save it to an environment variable. Run the following command:

$SERVICE_ACCOUNT_ISSUER =$(az connectedk8s show --n $aks_cluster_name --resource-group $resource_group_name --query "oidcIssuerProfile.issuerUrl" --output tsv)

Step 2: Create a Kubernetes service account and bind it to the Azure Managed Identity

First, create a managed identity. Run the az identity create command:

az identity create --name $MSIName --resource-group $resource_group_name --location $Location --subscription $AZSubscriptionID

Next, create variables for the managed identity's client ID:

$MSIId=$(az identity show --resource-group $resource_group_name --name $MSIName --query 'clientId' --output tsv)
$MSIPrincipalId=$(az identity show --resource-group $resource_group_name --name $MSIName --query 'principalId' --output tsv)

Create a Kubernetes service account

Create a Kubernetes service account and annotate it with the client ID of the managed identity created in the previous step:

az connectedk8s proxy -n $aks_cluster_name -g $resource_group_name

Open a new window. Copy and paste the following CLI commands:

$yaml = @" apiVersion: v1 kind: ServiceAccount metadata: annotations: azure.workload.identity/client-id: $MSIId name: $SERVICE_ACCOUNT_NAME namespace: $SERVICE_ACCOUNT_NAMESPACE "@ $yaml = $yaml -replace '\$MSIId', $MSIId ` -replace '\$SERVICE_ACCOUNT_NAME', $SERVICE_ACCOUNT_NAME ` -replace '\$SERVICE_ACCOUNT_NAMESPACE', $SERVICE_ACCOUNT_NAMESPACE $yaml | kubectl apply -f -

The following output shows successful creation of the service account:

serviceaccount/workload-identity-sa created

Step 3: Create a federated credential on the managed identity to trust the OIDC issuer

First, create a federated identity credential. Call the az identity federated-credential create command to create the federated identity credential between the managed identity, the service account issuer, and the subject. For more information about federated identity credentials in Microsoft Entra, see Overview of federated identity credentials in Microsoft Entra ID.

# Create a federated credential 

az identity federated-credential create --name $FedIdCredentialName --identity-name $MSIName --resource-group $resource_group_name --issuer $SERVICE_ACCOUNT_ISSUER --subject "system:serviceaccount:${SERVICE_ACCOUNT_NAMESPACE}:${SERVICE_ACCOUNT_NAME}" 

# Show the federated credential 

az identity federated-credential show --name $FedIdCredentialName --resource-group $resource_group_name --identity-name $MSIName

Note

After you add a federated identity credential, it takes a few seconds to propagate. Token requests made immediately afterward might fail until the cache refreshes. To prevent this issue, consider adding a brief delay after creating the federated identity credential.

Step 4: Deploy your application

When you deploy your application pods, the manifest should reference the service account created in the Create Kubernetes service account step. The following manifest shows how to reference the account, specifically the metadata\namespace and spec\serviceAccountName properties. Make sure to specify an image for image and a container name for containerName:

$image = "<image>"  # Replace <image> with the actual image name 
$containerName = "<containerName>"  # Replace <containerName> with the actual container name 

$yaml = @" 
apiVersion: v1 
kind: Pod 
metadata: 
  name: sample-quick-start 
  namespace: $SERVICE_ACCOUNT_NAMESPACE 
  labels: 
    azure.workload.identity/use: "true" # Required. Only pods with this label can use workload identity. 
spec: 
  serviceAccountName: $SERVICE_ACCOUNT_NAME 
  containers: 
    - image: $image 
      name: $containerName 
"@ 

# Replace variables within the YAML content 
$yaml = $yaml -replace '\$SERVICE_ACCOUNT_NAMESPACE', $SERVICE_ACCOUNT_NAMESPACE ` 
                -replace '\$SERVICE_ACCOUNT_NAME', $SERVICE_ACCOUNT_NAME 

# Apply the YAML configuration 
$yaml | kubectl apply -f -

Important

Ensure that the application pods using workload identity include the label azure.workload.identity/use: "true" in the pod spec. Otherwise the pods fail after they restart.

Example: Grant permissions to access Azure Key Vault

The instructions in this step describe how to access secrets, keys, or certificates in an Azure key vault from the pod. The examples in this section configure access to secrets in the key vault for the workload identity, but you can perform similar steps to configure access to keys or certificates.

The following example shows how to use the Azure role-based access control (Azure RBAC) permission model to grant the pod access to the key vault. For more information about the Azure RBAC permission model for Azure Key Vault, see Grant permission to applications to access an Azure key vault using Azure RBAC.

Create a key vault with purge protection and RBAC authorization enabled. You can also use an existing key vault if it is configured for both purge protection and RBAC authorization:

az keyvault create --name $KVName --resource-group $resource_group_name --location $Location --enable-purge-protection --enable-rbac-authorization

# retrieve the key vault ID for role assignment
$KVId=$(az keyvault show --resource-group $resource_group_name --name $KVName --query id --output tsv)

Assign the RBAC Key Vault Secrets Officer role to yourself so that you can create a secret in the new key vault. New role assignments can take up to five minutes to propagate and be updated by the authorization server.
```
az role assignment create --assignee-object-id $MSIPrincipalId --role "Key Vault Secrets Officer" --scope $KVId --assignee-principal-type ServicePrincipal
```

Create a secret in the key vault:

az keyvault secret set --vault-name $KVName --name $KVSecretName --value "Hello!"

Assign the Key Vault Secrets User role to the user-assigned managed identity that you created previously. This step gives the managed identity permission to read secrets from the key vault:
```
az role assignment create --assignee-object-id $MSIPrincipalId --role "Key Vault Secrets User" --scope $KVId --assignee-principal-type ServicePrincipal
```

Create an environment variable for the key vault URL:

$KVUrl=$(az keyvault show --resource-group $resource_group_name --name $KVName --query properties.vaultUri --output tsv)

Deploy a pod that references the service account and key vault URL:

$yaml = @" 
apiVersion: v1 
kind: Pod 
metadata: 
  name: sample-quick-start 
  namespace: $SERVICE_ACCOUNT_NAMESPACE 
  labels: 
    azure.workload.identity/use: "true" 
spec: 
  serviceAccountName: $SERVICE_ACCOUNT_NAME 
  containers: 
    - image: ghcr.io/azure/azure-workload-identity/msal-go 
      name: oidc 
      env: 
      - name: KEYVAULT_URL 
        value: $KVUrl 
      - name: SECRET_NAME 
        value: $KVSecretName 
  nodeSelector: 
    kubernetes.io/os: linux 
"@ 

# Replace variables within the YAML content 
$yaml = $yaml -replace '\$SERVICE_ACCOUNT_NAMESPACE', $SERVICE_ACCOUNT_NAMESPACE ` 
                -replace '\$SERVICE_ACCOUNT_NAME', $SERVICE_ACCOUNT_NAME ` 
                -replace '\$KVUrl', $KVUrl ` 
                -replace '\$KVSecretName', $KVSecretName 

# Apply the YAML configuration 
$yaml | kubectl --kubeconfig <path-to-aks-cluster-kubeconfig> apply -f -

Delete AKS Arc cluster

To delete the AKS Arc cluster, use the az aksarc delete command:

az aksarc delete -n $aks_cluster_name -g $resource_group_name

Note

There's a known issue when deleting an AKS Arc cluster with PodDisruptionBudget (PDB) resources: the deletion might fail to remove these PDB resources. Microsoft is aware of the problem and is working on a fix.

PDB is installed by default in workload identity-enabled AKS Arc clusters. To delete a workload identity enabled AKS Arc cluster, see the troubleshooting guide.

Next steps

In this article, you deployed a Kubernetes cluster and configured it to use a workload identity in preparation for application workloads to authenticate with that credential. Now you're ready to deploy your application and configure it to use the workload identity with the latest version of the Azure Identity client library.

Share via

Deploy and configure Workload Identity on an AKS enabled by Azure Arc cluster (preview)

Prerequisites

Export environment variables

Set the active subscription

Create a resource group

Step 1: Create an AKS Arc cluster with workload identity enabled

Save the OIDC issuer URL to an environment variable

Step 2: Create a Kubernetes service account and bind it to the Azure Managed Identity

Create a Kubernetes service account

Step 3: Create a federated credential on the managed identity to trust the OIDC issuer

Step 4: Deploy your application

Example: Grant permissions to access Azure Key Vault

Delete AKS Arc cluster

Next steps

Feedback

Additional resources