how use azure vault with managed identity on azure arc enabled k8s cluster.

Question

we have follow the given step to install and use azure vault with our on-prem cluster.

so we want to use azure managed identity with our on-prem cluster thats why we use azure arc

to connect our cluster to azure.

steps:

1. connect on-prem cluster to azure:

• az extenstion add --name connectedk8s
• az provider register -- namespace Microsoft.Kubernetes
• az provider register -- namespace Microsoft.KubernetesConfiguration
• az provider register -- namespace Microsoft.ExtendedLocation
• az connectedk8s connect --name <cluster-name> --resource-group <resource-group-name> --location <location>

2.create kubernetes serviceaccount and clusterrolebinding:

• kubectl create serviceaccount azure-arc-viewer -n azure-arc
• kubectl create clusterrolebinding azure-arc-viewer-binding --clusterrole=view --serviceaccount=azure-arc:azure-arc-viewer

3.genrate kubernetes token for azure arc:

• kubectl create token azure-arc-viewer
• kubectl get secret azure-arc-viewer-token -n azure-arc -o jsonpath="{.data.token}" | base64 --decode

cluster successfully connected to azure arc.

we have aleady create a sample secret in vault. now we want to use it in our on-prem cluster with managed identity.

4.Install Secrets Store CSI Driver & Azure Key Vault Provider:

az k8s-extension create \

--name akvsecretsprovider \

--cluster-name <your-arc-cluster-name> \

--resource-group <your-resource-group> \

--cluster-type connectedClusters \

--extension-type Microsoft.AzureKeyVaultSecretsProvider \

--config auto-rotate-secrets=true

5.verify the installation:

kubectl get po -n kube-system

pods are running successfully.

6.Grant Cluster Access to Azure Key Vault

• Get your cluster's Managed Identity ID:

az connectedk8s show --name <cluster-name> --resource-group <resource-group-name> --query identity.principalId -o tsv

create azure identity and assign role to it:

• Navigate to Your Key Vault
• Click on Access Control (IAM)
• Click on Add Role Assignment
• Select the Role as Key Vault Secrets User
• Assign access to: Choose Managed identity.
• Managed identity: Select Azure Arc enabled Kubernetes cluster
• Review + assign: Confirm and click Review + assign.

7.for testing we create secretproviderclass and test pod:

apiVersion: secrets-store.csi.x-k8s.io/v1

kind: SecretProviderClass

metadata:

  name: azure-kv-secrets

spec:

  provider: azure

  parameters:

    usePodIdentity: "false"

    useVMManagedIdentity: "true"           

    userAssignedIdentityID: ""              

    keyvaultName: <your-key-vault-name>

    objects: |

      array:

        - |

          objectName: my-secret            

          objectType: secret              

    tenantId: <your-azure-tenant-id>       

test pod:

apiVersion: v1

kind: Pod

metadata:

  name: nginx-secrets-test

spec:

  containers:

  - name: nginx

    image: nginx

    volumeMounts:

    - name: secrets-store

      mountPath: "/mnt/secrets"

      readOnly: true

  volumes:

    - name: secrets-store

      csi:

        driver: secrets-store.csi.k8s.io

        readOnly: true

        volumeAttributes:

          secretProviderClass: "azure-kv-secrets"

still its show the error.

`
if seVMManagedIdentity = true in SecretProviderClass

Normal Scheduled 2m11s default-scheduler Successfully assigned default/nginx-secrets-test to 10.171.248.85

Warning FailedMount 11s kubelet MountVolume.SetUp failed for volume "secrets-store" : rpc error: code = DeadlineExceeded desc = context deadline exceeded

if seVMManagedIdentity = false in SecretProviderClass
Normal Scheduled 16s default-scheduler Successfully assigned default/nginx-secrets-test to 10.171.249.13

Warning FailedMount 1s (x6 over 17s) kubelet MountVolume.SetUp failed for volume "secrets-store" : rpc error: code = Unknown desc = failed to mount secrets store objects for pod default/nginx-secrets-test, err: rpc error: code = Unknown desc = failed to mount objects, error: failed to create auth config, error: failed to get credentials, nodePublishSecretRef secret is not set

`

can someone help

Answer 1

Hello Akshay Dhumale,

I understand you're trying to use Managed Identity with the Secrets Store CSI Driver on an Azure Arc-enabled Kubernetes cluster, but it's not working. It's because Managed Identity is not currently supported for this setup.

Microsoft specifically mentions:

"Currently, the Secrets Store CSI Driver on Arc-enabled clusters can be accessed through a service principal."

You can check below screenshot from official Microsoft Document highlighting this:

To resolve the error, you'll need to use Service Principal instead of managed identity.

Register an application in Entra ID and create client secret in it like below:

Store these Service Principal credentials like app ID and secret value in Kubernetes Secret (secrets-store-creds) by creating it like this:

kubectl create secret generic secrets-store-creds --from-literal clientid="appId" --from-literal clientsecret="secretValue"

Make sure to assign "Key Vault Secrets User" role to this service principal under Azure Key Vault resource or create Vault access policy allowing access to secret permissions based on your key vault configuration.

Now, follow the remaining steps mentioned in this Microsoft Article:

Use Azure Key Vault Secrets Provider extension to fetch secrets into Azure Arc-enabled Kubernetes clusters - Azure Arc | Microsoft

Hope this helps!

---

Please do not forget to click "Accept the answer” and Yes wherever the information provided helps you, this can be beneficial to other community members.

If you have any other questions or still running into more issues, let me know in the "comments" and I would be happy to help you.