Secure access to Azure OpenAI from Azure Kubernetes Service (AKS)
In this article, you learn how to secure access to Azure OpenAI from Azure Kubernetes Service (AKS) using Microsoft Entra Workload ID. You learn how to:
- Enable workload identities on an AKS cluster.
- Create an Azure user-assigned managed identity.
- Create a Microsoft Entra ID federated credential.
- Enable workload identity on a Kubernetes Pod.
Note
We recommend using Microsoft Entra Workload ID and managed identities on AKS for Azure OpenAI access because it enables a secure, passwordless authentication process for accessing Azure resources.
Before you begin
- You need an Azure account with an active subscription. If you don't have one, create an account for free.
- This article builds on Deploy an application that uses OpenAI on AKS. You should complete that article before you begin this one.
- You need a custom domain name enabled on your Azure OpenAI account to use for Microsoft Entra authorization. For more information, see Custom subdomain names for Azure AI services.
Prerequisites
Use the Bash environment in Azure Cloud Shell. For more information, see Quickstart for Bash in Azure Cloud Shell.
If you prefer to run CLI reference commands locally, install the Azure CLI. If you're running on Windows or macOS, consider running Azure CLI in a Docker container. For more information, see How to run the Azure CLI in a Docker container.
If you're using a local installation, sign in to the Azure CLI by using the az login command. To finish the authentication process, follow the steps displayed in your terminal. For other sign-in options, see Sign in with the Azure CLI.
When you're prompted, install the Azure CLI extension on first use. For more information about extensions, see Use extensions with the Azure CLI.
Run az version to find the version and dependent libraries that are installed. To upgrade to the latest version, run az upgrade.
Enable Microsoft Entra Workload ID on an AKS cluster
The Microsoft Entra Workload ID and OIDC Issuer Endpoint features aren't enabled on AKS by default. You must enable them on your AKS cluster before you can use them.
Set the resource group name and AKS cluster resource group name variables.
# Set the resource group variable RG_NAME=myResourceGroup # Set the AKS cluster resource group variable AKS_NAME=$(az resource list --resource-group $RG_NAME --resource-type Microsoft.ContainerService/managedClusters --query "[0].name" -o tsv)
Enable the Microsoft Entra Workload ID and OIDC Issuer Endpoint features on your existing AKS cluster using the
az aks update
command.az aks update \ --resource-group $RG_NAME \ --name $AKS_NAME \ --enable-workload-identity \ --enable-oidc-issuer
Get the AKS OIDC Issuer Endpoint URL using the
az aks show
command.AKS_OIDC_ISSUER=$(az aks show --resource-group $RG_NAME --name $AKS_NAME --query "oidcIssuerProfile.issuerUrl" -o tsv)
Create an Azure user-assigned managed identity
Create an Azure user-assigned managed identity using the
az identity create
command.# Set the managed identity name variable MANAGED_IDENTITY_NAME=myIdentity # Create the managed identity az identity create \ --resource-group $RG_NAME \ --name $MANAGED_IDENTITY_NAME
Get the managed identity client ID and object ID using the
az identity show
command.# Get the managed identity client ID MANAGED_IDENTITY_CLIENT_ID=$(az identity show --resource-group $RG_NAME --name $MANAGED_IDENTITY_NAME --query clientId -o tsv) # Get the managed identity object ID MANAGED_IDENTITY_OBJECT_ID=$(az identity show --resource-group $RG_NAME --name $MANAGED_IDENTITY_NAME --query principalId -o tsv)
Get the Azure OpenAI resource ID using the
az resource list
command.AOAI_RESOURCE_ID=$(az resource list --resource-group $RG_NAME --resource-type Microsoft.CognitiveServices/accounts --query "[0].id" -o tsv)
Grant the managed identity access to the Azure OpenAI resource using the
az role assignment create
command.az role assignment create \ --role "Cognitive Services OpenAI User" \ --assignee-object-id $MANAGED_IDENTITY_OBJECT_ID \ --assignee-principal-type ServicePrincipal \ --scope $AOAI_RESOURCE_ID
Create a Microsoft Entra ID federated credential
Set the federated credential, namespace, and service account variables.
# Set the federated credential name variable FEDERATED_CREDENTIAL_NAME=myFederatedCredential # Set the namespace variable SERVICE_ACCOUNT_NAMESPACE=default # Set the service account variable SERVICE_ACCOUNT_NAME=ai-service-account
Create the federated credential using the
az identity federated-credential create
command.az identity federated-credential create \ --name ${FEDERATED_CREDENTIAL_NAME} \ --resource-group ${RG_NAME} \ --identity-name ${MANAGED_IDENTITY_NAME} \ --issuer ${AKS_OIDC_ISSUER} \ --subject system:serviceaccount:${SERVICE_ACCOUNT_NAMESPACE}:${SERVICE_ACCOUNT_NAME}
Use Microsoft Entra Workload ID on AKS
To use Microsoft Entra Workload ID on AKS, you need to make a few changes to the ai-service
deployment manifest.
Create a ServiceAccount
Get the kubeconfig for your cluster using the
az aks get-credentials
command.az aks get-credentials \ --resource-group $RG_NAME \ --name $AKS_NAME
Create a Kubernetes ServiceAccount using the
kubectl apply
command.kubectl apply -f - <<EOF apiVersion: v1 kind: ServiceAccount metadata: annotations: azure.workload.identity/client-id: ${MANAGED_IDENTITY_CLIENT_ID} name: ${SERVICE_ACCOUNT_NAME} namespace: ${SERVICE_ACCOUNT_NAMESPACE} EOF
Enable Microsoft Entra Workload ID on the Pod
Set the Azure OpenAI resource name, endpoint, and deployment name variables.
# Get the Azure OpenAI resource name AOAI_NAME=$(az resource list \ --resource-group $RG_NAME \ --resource-type Microsoft.CognitiveServices/accounts \ --query "[0].name" -o tsv) # Get the Azure OpenAI endpoint AOAI_ENDPOINT=$(az cognitiveservices account show \ --resource-group $RG_NAME \ --name $AOAI_NAME \ --query properties.endpoint -o tsv) # Get the Azure OpenAI deployment name AOAI_DEPLOYMENT_NAME=$(az cognitiveservices account deployment list \ --resource-group $RG_NAME \ --name $AOAI_NAME \ --query "[0].name" -o tsv)
Redeploy the
ai-service
with the ServiceAccount and theazure.workload.identity/use
annotation set totrue
using thekubectl apply
command.kubectl apply -f - <<EOF apiVersion: apps/v1 kind: Deployment metadata: name: ai-service spec: replicas: 1 selector: matchLabels: app: ai-service template: metadata: labels: app: ai-service azure.workload.identity/use: "true" spec: serviceAccountName: $SERVICE_ACCOUNT_NAME nodeSelector: "kubernetes.io/os": linux containers: - name: ai-service image: ghcr.io/azure-samples/aks-store-demo/ai-service:latest ports: - containerPort: 5001 env: - name: USE_AZURE_OPENAI value: "True" - name: USE_AZURE_AD value: "True" - name: AZURE_OPENAI_DEPLOYMENT_NAME value: "${AOAI_DEPLOYMENT_NAME}" - name: AZURE_OPENAI_ENDPOINT value: "${AOAI_ENDPOINT}" resources: requests: cpu: 20m memory: 50Mi limits: cpu: 50m memory: 128Mi EOF
Test the application
Verify the new pod is running using the
kubectl get pods
command.kubectl get pods --selector app=ai-service -w
Get the pod logs using the
kubectl logs
command. It may take a few minutes for the pod to initialize.kubectl logs --selector app=ai-service -f
The following example output shows the app has initialized and is ready to accept requests. The first line suggests the code is missing configuration variables. However, the Azure Identity SDK handles this process and sets the
AZURE_CLIENT_ID
andAZURE_TENANT_ID
variables.Incomplete environment configuration. These variables are set: AZURE_CLIENT_ID, AZURE_TENANT_ID INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:5001 (Press CTRL+C to quit)
Get the pod environment variables using the
kubectl describe pod
command. The output demonstrates that the Azure OpenAI API key no longer exists in the Pod's environment variables.kubectl describe pod --selector app=ai-service
Open a new terminal and get the IP of the store admin service using the following
echo
command.echo "http://$(kubectl get svc/store-admin -o jsonpath='{.status.loadBalancer.ingress[0].ip}')"
Open a web browser and navigate to the IP address from the previous step.
Select Products. You should be able to add a new product and get a description for it using Azure OpenAI.
Next steps
In this article, you learned how to secure access to Azure OpenAI from Azure Kubernetes Service (AKS) using Microsoft Entra Workload ID.
For more information on Microsoft Entra Workload ID, see Microsoft Entra Workload ID.
Azure Kubernetes Service