Managing Github URLs for Azure Resource Manager Template Deployments
Have you ever tried the following: You push an Azure Resource Manager template to GitHub and try to deploy using the GitHub URL. Unfortunately, you had a bug or typo in your template and you get an error. You quickly fix the error and push a revised version and try again, but the Resource Manager is picking up a cached version of your template and you get the error again. Annoying, right? In this blog, I will provide a bit of help on how to avoid that.
If you are deploying cloud computing resources in Azure, it is recommended that you use Azure Resource Manager Templates to make the process easier to reproduce, document, and share. It is also a great idea to put your templates in some sort of source code repository that will allow you to track changes, etc. GitHub is a popular choice. As your templates grow in complexity they will be referencing other templates and artifacts (e.g., configuration scripts) and these dependencies should also be in the source code repository. It is tempting to put absolute URL references to dependencies and many published templates are littered with those https://raw.githubusercontent.com URLs that make it hard to clone and revise templates in repositories. If you manage multiple branches and versions of your template it becomes very problematic to edit the links as you work through revisions. A better way (in my opinion) is to never put an absolute https://raw.githubusercontent.com link into your template. Not everybody agrees with this, but in my experience, it is far easier to manage when the links are always relative to the URL of template currently being deployed. This URL is available in the template with the deployment().properties.templateLink.uri construct. While I am a firm believer that eliminating the absolute URLs is the right thing to do, it creates a new problem that you have to deal with. If you reference deployment().properties.templateLink.uri you can no longer deploy the template directly from the command line, e.g. with something like:
[ps]
New-AzureRmResourceGroupDeployment -Name myDeploy -TemplateFile azuredeploy.json ...
[/ps]
You will get an error that deployment().properties.templateLink.uri is not defined, which makes sense. The solution to this problem is, of course, to push the template to a branch in your GitHub repository and deploy with:
[ps]
New-AzureRmResourceGroupDeployment -Name myDeploy `
-TemplateUri https://raw.githubusercontent.com/\<USER>/<REPO>/<BRANCH>/<FOLDER>/azuredeploy.json
[/ps]
Problem solved, right? Well not so fast because what happens is that as you develop your template, you will need to make multiple revisions to it and will need deploy (or at least validate) it over and over again. When you reference a GitHub branch as in the example above, you will have a problem with the Azure Resource Manager picking up a cached version of your template. To be specific, you will upload a template and try to validate or deploy and you get an error. You quickly fix the error and push a new version of the template and try again, but the Resource Manager is still picking up the old one. Annoying. You could of course go grab a coffee and wait for the cache to clear, but imagine you have to do this 20 times in a day and you would quickly reach your caffeine limit. I am sure you get the idea and you probably know that the solution is to reference a specific commit in the URL. So instead of:
[plain]
https://raw.githubusercontent.com/\<USER>/<REPO>/mybranch/azuredeploy.json
[/plain]
You would reference:
[plain]
https://raw.githubusercontent.com/\<USER>/<REPO>/ed29c53f741f683d8febd7f5eaaf1890bca9365a/azuredeploy.json
[/plain]
This is actually a great solution, and it guarantees that you are referring to the exact revision that you just pushed. You can always find the latest commit with:
[ps]
git rev-parse HEAD
[/ps]
However, if you have to do this many times, it becomes a bit tedious (and error prone) to have to figure out the SHA-1 hash of the commit and then copy paste that into the TemplateUri for every time. To assist with this, I have made a simple function called Get-GitHubRawPath. You can find it here on GitHub. It is also part of a small Powershell Module called HansenAzurePS that you can find on the Powershell Gallery. If you would like to install this module, you can use:
[ps]
Install-Module HansenAzurePS
[/ps]
In addition to the Get-GitHubRawPath function. There are also a few other useful tools, e.g. functions creating Web App AAD registrations and for deleting Azure Recovery Services Vaults that have been discussed in previous blog posts.
The way that you use the function is by navigating to the folder within the git repository containing the template and using a command like:
[ps]
Get-GitHubRawPath -File <FILE NAME>
[/ps]
The function will return the correct https://raw.githubusercontent.com link with an absolute commit hash. So for example, if you have a git repository with a template called azuredeploy.json and you have just pushed a revision to GitHub and now you would like to deploy the version of the template corresponding to this latest revision (i.e., HEAD of your current branch), then you can deploy with:
[ps]
New-AzureRmResourceGroupDeployment -Name myDeploy `
-TemplateUri $(Get-GitHubRawPath -File .\azuredeploy.json)
[/ps]
The Get-GitHubRawPath command has a lot of built in features that you come in handy when working with revisions of templates. First of all, you can invoke the command with a relative path to a file anywhere in the repo, and the correct URL will be constructed taking the root of the git repository into consideration. Say you issue the command above from a sub-sub folder of your repo, the correct full URL for GitHub will be generated. You can also add information about a specific commit or branch, e.g. if you wanted to deploy the version at the head of the branch "myBranch":
[ps]
New-AzureRmResourceGroupDeployment -Name myDeploy `
-TemplateUri $(Get-GitHubRawPath -Revision myBranch -File .\azuredeploy.json)
[/ps]
Or if you know the first few digits of the SHA-1 hash you can even use that:
[ps]
New-AzureRmResourceGroupDeployment -Name myDeploy `
-TemplateUri $(Get-GitHubRawPath -Revision 177336a -File .\azuredeploy.json)
[/ps]
And it will locate the correct commit (assuming there is no ambiguity) and generate the full path. These capabilities for referencing specific branches and commits come in handy if you are trying to track down when a recently discovered bug was introduced.
There is nothing terribly fancy or elaborate about the code for the Get-GitHubRawPath function and it should be pretty easy to modify if you wanted to include more or different functionality. It is one of those little tools that has saved me a lot of time and irritation while working with templates and I hope you find it useful too.
Let me know if you have questions/suggestions/comments.
Comments
- Anonymous
January 16, 2018
Great Post Michael! I've been using it over the last few days while working on a template for a customer, works as advertised. Thank you!- Anonymous
January 16, 2018
Thanks Jason. Glad somebody else can find some use for it too.
- Anonymous