Rotate service principal credentials for your Azure Red Hat OpenShift (ARO) Cluster

Article
02/24/2023

The article provides the necessary detail to rotate service principal credentials in Azure Red Hat OpenShift clusters (ARO).

Before you begin

The article assumes that there is an existing ARO cluster with the latest updates applied.

The minimum Azure CLI requirements to rotate service principal credentials within an ARO cluster is 2.24.0.

To check the version of Azure CLI run:

# Azure CLI version
az --version

To install or upgrade Azure CLI please follow Install Azure CLI.

The following instructions use bash syntax.

Service principal credential rotation

Important

Service principal credential rotation can take upwards of 2 hours depending on cluster state.

Service principal credential rotation has two methods:

Automated service principal credential rotation
User provided client-id and client-secret service principal credential rotation

Automated service principal credential rotation

Important

Automated service principal credential rotation requires the ARO cluster to be created with Azure CLI version 2.24.0 or greater.

Automated service principal credential rotation will check if the service principal exists and rotate or create a new service principal.

Automatically rotate service principal credentials with the following command:

# Automatically rotate service principal credentials
az aro update --refresh-credentials --name MyManagedCluster --resource-group MyResourceGroup

User provided client-id and client-secret service principal credential rotation

Manually rotate service principal credentials with user provided client-id and client-secret with the following instructions:

Retrieve the service principal clientId (--client-id) and set it as SP_ID environment variable.

# Retrieve the service principal clientId
SP_ID=$(az aro show --name MyManagedCluster --resource-group MyResourceGroup \
    --query servicePrincipalProfile.clientId -o tsv)

Generate a new secure secret (--client-secret) for the service principal using the SP_ID variable above. Store the new secure secret as SP_SECRET environment variable.

# Generate a new secure secret for the service principal
SP_SECRET=$(az ad sp credential reset --id $SP_ID --query password -o tsv)

Rotate service principal credentials using the above environment variables.

# Rotate service principal credentials
az aro update --client-id $SP_ID --client-secret $SP_SECRET \
    --name MyManagedCluster --resource-group MyResourceGroup

Troubleshoot

Service principal expiration date

Service principal credentials have a set expiration date of a year and should be rotated within that given timeframe.

If the expiration date has passed the following errors are possible:

Failed to refresh the Token for request to MyResourceGroup StatusCode=401
Original Error: Request failed. Status Code = '401'.
[with]
Response body: {"error":"invalid_client","error_description": The provided client secret keys are expired.
[or]
Response body: {"error":"invalid_client","error_description": Invalid client secret is provided.

To check the expiration date of service principal credentials run the following:

# Service principal expiry in ISO 8601 UTC format
SP_ID=$(az aro show --name MyManagedCluster --resource-group MyResourceGroup \
    --query servicePrincipalProfile.clientId -o tsv)
az ad app credential list --id $SP_ID --query "[].endDateTime" -o tsv

If the service principal credentials are expired please update using one of the two credential rotation methods.

Cluster AAD application contains a client secret with an empty description

When using automated service principal credential rotation the following error occurs:

$ az aro update --refresh-credentials --name MyManagedCluster --resource-group MyResourceGroup

Cluster AAD application contains a client secret with an empty description.
Please either manually remove the existing client secret and run `az aro update --refresh-credentials`,
or manually create a new client secret and run `az aro update --client-secret <ClientSecret>`.

The cluster has not been created using Azure CLI 2.24.0 or greater. Use the user provided client-id and client-secret service principal credential rotation method instead.

Azure CLI ARO update help

For more details please see the Azure CLI ARO update help command:

# Azure CLI ARO update help
az aro update -h

Share via