Share via

Enhancing the security of Azure Automation Webhooks in an Azure DevSecOps context

  • Article

Hi,

 

Webhooks are a very convenient way to integrate APIs in general and to call Azure Automation runbooks but while they are very useful and easy to work with, they raise some security concerns. To give a concrete example, if you create a webhook against a runbook that leverages Azure Automation Hybrid Workers, causing this runbook to execute against on-premises machines and/or within your network boundaries, you might want to make sure that the webhook consumer is well eligible to do so.

The problem is that with Azure Automation runbook webhooks (like with many other systems), there is no way to control whether the caller is an expected one. The possessor of a webhook URL can simply start using it, from whereever location and whoever it is, or whatever the application is. Azure Automation is serverless and public facing so anyone in the world having a webhook URL can use it.

This somehow resembles Access Tokens, if you gain access to one, you can consider them as keys, you can open the door even if you're not the owner of the house :). The difference is that, with AzureAD generated tokens, the default lifetime is rather short (1h) while the default lifetime of a webhook is 1 year...

In security, the danger is to use the same key (certificate, storage key, azure app secret, etc...) without rotating it  frequently, because the likelihood a key could be compromised/leaked increases over time and it also allows an attacker to secretly abuse your environment over a longer period. Do not underestimate insider's potential nuisance as they often have access to a lot of information they could reuse against you once they leave the company. Therefore, the key (no word game here :)) is to have short-lived secrets. When it comes to Azure Automation Runbook Webhooks, I'd recommend one-time webhooks. So, the idea is to generate the webhook on the fly and delete it right after the first use.

I talked about this with one of my customers who argued that this was indeed interesting but required the webhook creator to have contributor permissions over the Automation Account and that was a bit too much. I'm still convinced that it's better to grant a system (not a person) broader permissions rather than letting webhook URLs here and there and be potentially stolen and re-used by non-eligible parties. Plus, I prefer to revoke (or renew the app credentials) permissions granted to a single system rather than invalidating all the webhooks one by one for every possible runbook that might be called form Azure DevOps...Nevertheless, I reconsidered my approach and found what I think is a good trade-off between potentially uncontrolled webhook URLs and granting too much permissions to a system. With my one-time webhook URLs, I know that the likelyhood for a URL to be disclosed is almost non-existent and that even if this happens, it will be very very short-lived since it is deleted once the runbook's job is created. Regarding the system that creates the webhook, I managed to narrow down its permissions to the bare minimum (more info later). This means that the residual risk is the leakage/abuse of the systems'credentials to unexpectedly create webhooks on runbooks. But here, we come to my previous point, in such a case, you could rotate or invalidate the system's SPN's permissions at once.

So, let's come to the meat of it! As you read from this post's title, the whole idea targets a DevSecOps way of working with automated deployments where you might call Azure Automation Runbooks from Azure DevOps using Webhooks. So, instead of recording webhooks in Automation and define them in your release definitions, my idea is to create a Service endpoint in Azure DevOps, grant it a custom-role (I'll come to this) over the Automation Runbooks (note that I said runbooks not the entire Automation Account) and use that SPN in your build/release tasks to create one-time webhooks and call runbooks. The final result could look like this:

So the steps are:

  • Create an Azure App of type WEBAPI.
  • Use the resulting Client ID & Client Secret to configure the Azure DevOps Service endpoint (of type ARM)
  • Grant the custom role (see below) to the SPN
  • Use this SPN in your build/release tasks to call Automation Runbooks with a safer way to handle webhooks

So, the good news is that I already made the hard work, meaning a custom VSTS extension (screenshot) that makes use of the SPN to call automation runbooks and create webhooks on the fly and the custom RBAC role to associate to the SPN you create for Azure DevOps. To create my custom role, I started from the built-in Automation Operator role (ability to start, stop, suspend and resume jobs) to which I added an extra permission over creation and deletion of webhooks.

The Azure Tools extension may be found on the public marketplace  while its source code plus the custom role, may be found in its associated GitHub repo.

If this true DevSecOps way of working makes sense for you, the only thing left to you is the configuration of the SPN and the addition of the custom role to your environment.