Use Azure Key Vault secrets in your Pipeline

Azure DevOps Services | Azure DevOps Server 2022 - Azure DevOps Server 2019

With Azure Key Vault, you can securely store and manage your sensitive information such as passwords, API keys, certificates, etc. using Azure Key Vault, you can easily create and manage encryption keys to encrypt your data. Azure Key Vault can also be used to manage certificates for all your resources. In this article, you'll learn how to:

• Create an Azure Key Vault.
• Configure your Key Vault permissions.
• Create a new service connection.
• Query for secrets from your Azure Pipeline.

Prerequisites

• An Azure DevOps organization. Create one for free if you don't already have one.
• Your own project. Create a project if you don't already have one.
• Your own repository. Create a new Git repo if you don't already have one.
• An Azure subscription. Create a free Azure account if you don't already have one.

Create a Key Vault

Azure portal

1. Sign in to the Azure portal, and then select Create a resource.

2. Under Key Vault, select Create to create a new Azure Key Vault.

3. Select your Subscription from the dropdown menu, and then select an existing Resource group or create a new one. Enter a Key vault name, select a Region, choose a Pricing tier, and select Next if you want to configure additional properties. Otherwise, select Review + create to keep the default settings.

4. Once the deployment is complete, select Go to resource.

Azure CLI

1. First, set your default region and Azure subscription:

   • Set default subscription:

   az account set --subscription <your_subscription_name_or_subscription_ID>
   

   • Set default region:

   az config set defaults.location=<your_region>
   

2. Create a new resource group to host your Azure Key Vault. A resource group is a container that holds related resources for an Azure solution:

   az group create --name <your-resource-group>
   

3. Create a new Azure Key Vault:

   az keyvault create \
     --name <your-key-vault-name> \
     --resource-group <your-resource-group>
   

Set up authentication

Managed Identity

Create a user-assigned managed identity

1. Sign in to the Azure portal, then search for the Managed Identities service in the search bar.

2. Select Create, and fill out the required fields as follows:

   • Subscription: Select your subscription from the dropdown menu.
   • Resource group: Select an existing resource group or create a new one.
   • Region: Select a region from the dropdown menu.
   • Name: Enter a name for your user-assigned managed identity.

3. Select Review + create when you're done.

4. Once the deployment is complete, select Go to resource, then copy the Subscription and Client ID values to use in upcoming steps.

5. Navigate to Settings > Properties, and copy your managed identity's Tenant ID value for later use.

Set up key vault access policies

1. Navigate to Azure portal, and use the search bar to find the key vault you created earlier.

2. Select Access policies, then select Create to add a new policy.

3. Under Secret permissions, select Get and List checkboxes.

4. Select Next, then paste the Client ID of the managed identity you created earlier into the search bar. Select your managed identity.

5. Select Next, then Next once more.

6. Review your new policies, and then select Create when you're done.

Create a service connection

1. Sign in to your Azure DevOps organization, and then navigate to your project.

2. Select Project settings > Service connections, and then select New service connection to create a new service connection.

3. Select Azure Resource Manager, then select Next.

4. For Identity Type, select Managed identity from the dropdown menu.

5. For Step 1: Managed identity details, fill out the fields as follows:

   • Subscription for managed identity: Select the subscription containing your managed identity.

   • Resource group for managed identity: Select the resource group hosting your managed identity.

   • Managed Identity: Select your managed identity from the dropdown menu.

6. For Step 2: Azure Scope, fill out the fields as follows:

   • Scope level for service connection: Select Subscription.

   • Subscription for service connection: Select the subscription your managed identity will access.

   • Resource group for Service connection: (Optional) Specify to limit managed identity access to one resource group.

7. For Step 3: Service connection details:

   • Service connection name: Provide a name for your service connection.

   • Service Management Reference: (Optional) Context information from an ITSM database.

   • Description: (Optional) Add a description.

8. In Security, select the Grant access permission to all pipelines checkbox to allow all pipelines to use this service connection. If you don't select this option, you must manually grant access to each pipeline that uses this service connection.

9. Select Save to validate and create the service connection.

   

Service Principal

Create a service principal

In this step, we will create a new service principal in Azure, enabling us to query our Azure Key Vault from Azure Pipelines.

1. Navigate to Azure portal, then select the >_ icon from the menu bar to open the Cloud Shell.

2. Select PowerShell or leave it as Bash based on your preference.

3. Run the following command to create a new service principal:

   az ad sp create-for-rbac --name YOUR_SERVICE_PRINCIPAL_NAME
   

4. Your output should match the example below. Be sure to copy the output of your command, as you will need it to create the service connection in the upcoming step.

   {
     "appId": "00001111-aaaa-2222-bbbb-3333cccc4444",
     "displayName": "MyServicePrincipal",
     "password": "***********************************",
     "tenant": "aaaabbbb-0000-cccc-1111-dddd2222eeee"
   }
   

Create a service connection

1. Sign in to your Azure DevOps organization, and then navigate to your project.

2. Select Project settings, and then select Service connections.

3. Select New service connection, select Azure Resource Manager, and then select Next.

4. Select Service principal (manual), and then select Next.

5. For Identity Type, select App registration or managed identity (manual) from the dropdown menu.

6. For Credential*, select Workload identity federation.

7. Provide a name for your service connection, and then select Next.

8. Copy the Issuer and the Subject identifier as we will need it in the next step.

9. Select Azure Cloud for Environment, and Subscription for the Subscription scope.

10. Enter your Azure Subscription ID and Subscription name.

11. For Authentication, paste your service principal's Application (client) ID and Directory (tenant) ID

12. Under Security, select the Grant access permission to all pipelines checkbox to allow all pipelines to use this service connection. If you don't select this option, you must manually grant access to each pipeline that uses this service connection.

13. Leave this open, you'll return to verify and save once you've (1) created the federated credential in Azure and (2) granted your service principal Read access at the subscription level.

    

Create a federated credential in Azure

1. Navigate to Azure portal, then enter your service principal's ClientID in the search bar, and then select your Application.

2. Under Manage, select Certificates & secrets > Federated credentials.

3. Select Add credential, and then for Federated credential scenario, select Other issuer.

4. For Issuer, paste the Issuer you copied from your service connection earlier.

5. For Subject identifier, paste the Subject identifier you copied from your service connection earlier.

6. Provide a Name for your federated credential, and then select Add when you're done.

   

Add role assignment to your subscription

Before you can verify the connection, you need to grant the service principal Read access at the subscription level:

1. Navigate to Azure portal

2. Under Azure service, select Subscriptions, and then find and select your subscription.

3. Select Access control (IAM), and then select Add > Add role assignment.

4. Select Reader under the Role tab, and then select Next.

5. Select User, group, or service principal, and then select Select members.

6. In the search bar, paste your service principal's Object ID, select it, then click on the Select button.

7. Select Review + assign, review your settings, and then select Review + assign once more to confirm your choices and add the role assignment.

8. Once the role assignment is added. go back to your service connection (in Azure DevOps) to finally select Verify and Save to save your service connection.

Configure Key Vault access policies

1. Navigate to Azure portal, find the key vault you created earlier, and then select Access policies.

2. Select Create, and then under Secret permissions add the Get and List permissions, and then select Next.

3. Under Principal, paste your service principal's Object ID, select it and then select Next.

4. Select Next once more, review your settings, and then select Save when you're done.

Query and use secrets in your pipeline

Using the Azure Key Vault task we can fetch the value of our secret and use it in subsequent tasks in our pipeline. One thing to keep in mind is that secrets must be explicitly mapped to env variable as shown in the example below.

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: AzureKeyVault@1
  inputs:
    azureSubscription: 'SERVICE_CONNECTION_NAME'
    KeyVaultName: 'KEY_VAULT_NAME'
    SecretsFilter: '*'

- bash: |
    echo "Secret Found! $MY_MAPPED_ENV_VAR"        
  env:
    MY_MAPPED_ENV_VAR: $(SECRET_NAME)

The output from the last bash command should look like this:

Secret Found! ***

Note

If you want to query for multiple secrets from your Azure Key Vault, use the SecretsFilter argument to pass a comma-separated list of secret names: 'secret1, secret2'.

Related content

• Manage service connections
• Define variables
• Publish Pipeline Artifacts