Scripts-to-Tools: Automate Health Monitoring Alert Rules in the Cloud with PowerShell and the Azure Service Management REST API
After successfully provisioning new IaaS virtual machines or PaaS cloud services in Microsoft Azure, the focus often turns to workload monitoring for ensuring the continued health of the solutions we've deployed. Microsoft Azure includes native monitoring and email alerting capabilities for deployed workloads, and you can certainly extend monitoring to more granular levels with additional tools, such as Azure Automation, Application Insights, System Center 2012 R2 Operations Manager, or other 3rd party tools like New Relic.
However, manually configuring monitoring alert rules for a large number of workloads can take a lot of time. Unfortunately, the Azure PowerShell module doesn't currently provide direct scripting support to automate alert rule definition, BUT ... the Azure Service Management REST API does provide this capability, and we can easily leverage that API via PowerShell with a bit of creativity!
In this article, we'll step through the process of creating our own PowerShell function, named New-AzureAlert, to help us automate the provisioning of new Azure monitoring alert rules using the Azure Service Management REST API.
Getting Started
Before getting started in this article, you'll first need a few items:
- An active Microsoft Azure subscription. You can sign-up for a FREE Azure subscription trial if you don't currently have one.
- Microsoft Azure PowerShell module. Be sure to download and install the latest version to get all the most recent updates.
- At least one IaaS virtual machine or PaaS cloud service deployed to your Azure subscription. You can learn more about the steps involved in deploying workloads on Azure via our Early Experts program, and via our Step-by-Step Cloud Lab guides.
- Familiarity with the basics of building Powershell functions. If you need a quick primer, check out my prior Scripts-to-Tools article where I stepped through the process of building re-usable tools via PowerShell functions.
After you've collected everything you'll need to get started, let's proceed to the next step ...
Where's my Certificate?
To authenticate to the Azure Service Management REST API endpoints, we'll be passing the subscription ID and management certificate for your Azure subscription. To obtain your subscription ID and management certificate, you can use the following snippet of code:
# Download Azure Publish Settings File
Get-AzurePublishSettingsFile
# Import Azure Publish Settings File
Import-AzurePublishSettingsFile 'path-to-publish-settings-file'
# Get your Azure subscription ID and certificate
$subscriptionName = 'your-subscription-name'
$subscription = Get-AzureSubscription `
-SubscriptionName $subscriptionName `
-ExtendedDetails$certificate = $subscription.Certificate
$subscriptionId = $subscription.SubscriptionId
Tip! If you've previously authenticated to your Azure subscription via Azure AD credentials using the Add-AzureAccount cmdlet, you may first need to remove the Azure subscription information cached in your local subscription data file by using the Remove-AzureSubscription cmdlet. If you do not perform this step, the management certificate may not be properly returned when calling the Get-AzureSubscription cmdlet above.
Now that you've run through the lines above, you should have your Azure subscription ID stored in the $subscriptionId variable, and your management certificate should be stored in the $certificate variable.
How do we Function?
To make it super-easy to add new monitoring alerts via the Azure Service Management REST API, let's define a new function in PowerShell named New-AzureAlert. When defining this function, we'll include input parameters for each of the key values needed to define a new monitoring alert in an Azure subscription. We'll also include comment-based help, so that others who are calling this function can easily learn how to use it via the Get-Help cmdlet.
Tip! See my prior Scripts-to-Tools article for a complete walk-through on each of the components of a PowerShell function.
Here's the code that we'll use to define the New-AzureAlert function:
function New-AzureAlert {
<#
.SYNOPSIS
New-AzureAlert defines a new monitoring alert to an existing Cloud
Service deployment.
.DESCRIPTION
New-AzureAlert defines a new monitoring alert to an existing Cloud
Service deployment
for IaaS Virtual Machines or PaaS Web/Worker roles.
For demonstration purposes only.
No support or warranty is supplied or inferred.
Use at your own risk.
.PARAMETER subscriptionId
The Id of the Azure Subscription in which the Cloud Service is
deployed
.PARAMETER certificate
Certificate used for authenticating to Azure subscription Id
.PARAMETER cloudServiceName
The name of an existing Cloud Service, as reported
by Get-AzureService
.PARAMETER deploymentName
The name of an existing deployment within a Cloud Service, as
reported by Get-AzureDeployment
.PARAMETER roleName
The name of an existing role within a Cloud Service deployment, as
reported by Get-AzureDeployment
.PARAMETER alertName
The name for the new Alert to be added.
The name can contain only letters, numbers, commas, and periods.
The name can be up to 32 characters long.
.PARAMETER alertDescription
Optional description for the new Alert to be added.
The description can contain only letters, numbers, commas, and
periods.
The description can be up to 128 characters long.
.PARAMETER metricName
Name of the metric on which to associate an alert rule.
Valid metricName values include:
"Percentage CPU"
"Disk Read Bytes/Sec"
"Disk Write Bytes/Sec"
"Network In"
"Network Out"
.PARAMETER metricWindowSize
Rolling timeframe across which metric should be evaluated against
threshold.
Valid metricWindowSize values include:
"PT5M" (5 min)
"PT15M" (15 min)
"PT30M" (30 min)
"PT45M" (45 min)
"PT60M" (60 min)
.PARAMETER metricOperator
Operator used to compare metricName against metricThreshold.
Value metricOperator values include:
"GreaterThan"
"GreaterThanOrEqual"
"LessThan"
"LessThanOrEqual"
.PARAMETER metricThreshold
Numeric value to which metricName is compared to determine if alert
should be issued.
.PARAMETER alertAdmins
Indicates whether subscription administrators and co-administrators
should be alerted via email.
Valid alertAdmins values include: True, False
.PARAMETER alertOther
Email address for additional application administrator that should
be alerted via email.
.INPUTS
Parameters above.
.OUTPUTS
json output for alert rule configuration that was successfully
provisioned.
.NOTES
Version: 1.0
Creation Date: Nov 8, 2014
Author: Keith Mayer ( https://KeithMayer.com )
Change: Initial function development
#>[CmdletBinding()]
param
(
[string]$subscriptionId,
[object]$certificate,
[string]$cloudServiceName,
[string]$deploymentName,
[string]$roleName,
[string]$alertName,
[string]$alertDescription,
[string]$metricName,
[string]$metricWindowSize,
[string]$metricOperator,
[decimal]$metricThreshold,
[boolean]$alertAdmins,
[string]$alertOther
)begin {
$requestHeader = @{
"x-ms-version" = "2013-10-01";
"Accept" = "application/json"
}
$contentType = "application/json;charset=utf-8"
$alertEnabled = "true"if ($alertAdmins) {
$alertAdminsValue = "true"
} else {
$alertAdminsValue = "false"
}
}process {
$alertId = ([GUID]::NewGuid()).Guid
$alertManagementUri =
" https://management.core.windows.net/ $subscriptionId
/services/monitoring/alertrules/$alertID"$alertRequest = @"
{
"Id": "$alertID",
"Name": "$alertName",
"IsEnabled": $alertEnabled,
"Condition": {
"odata.type": "Microsoft.WindowsAzure.Management.Monitoring.Alerts.Models.
ThresholdRuleCondition",
"DataSource": {
"odata.type": "Microsoft.WindowsAzure.Management.Monitoring.Alerts.Models.
RuleMetricDataSource",
"ResourceId": "/hostedservices/$cloudServiceName/deployments/$deploymentName/roles/$roleName",
"MetricNamespace": "",
"MetricName": "$metricName"
},
"Operator": "$metricOperator",
"Threshold": $metricThreshold,
"WindowSize": "$metricWindowSize"
},
"Actions": [
{
"odata.type": "Microsoft.WindowsAzure.Management.Monitoring.Alerts.Models.RuleEmailAction",
"SendToServiceOwners": $alertAdminsValue,
"CustomEmails": [
"$alertOther"
]
}
]
}
"@[byte[]]$requestBody =
[System.Text.Encoding]::UTF8.GetBytes($alertRequest)$alertResponse = Invoke-RestMethod `
-Uri $alertManagementUri `
-Certificate $certificate `
-Method Put `
-Headers $requestHeader `
-Body $requestBody `
-ContentType $contentType}
end {
$alertListUri =
" https://management.core.windows.net/ $subscriptionID
/services/monitoring/alertrules"$alerts = Invoke-RestMethod `
-Uri $alertListUri `
-Certificate $certificate `
-Method Get `
-Headers $requestHeader `
-ContentType $contentType$alerts.Value | ConvertTo-Json
}
}
Note that in this function definition, the requests for defining and confirming a new monitoring alert are submitted to the Azure Service Management REST API by calling the Invoke-RestMethod cmdlet in the highlighted code blocks above. This cmdlet was introduced as a new feature in PowerShell 4.0.
Alert! Alert!
Our New-AzureAlert function is defined and ready to be used! We can easily call this function to define each monitoring alert as part of a larger provisioning script.
$alertRule = New-AzureAlert `
-alertName "High CPU" `
-alertDescription "Higher than 80% CPU utilization" `
-subscriptionId $subscriptionId `
-certificate $certificate `
-cloudServiceName "insert-cloud-service-name" `
-deploymentName "insert-deployment-name" `
-roleName "insert-VM-role-name" `
-metricName "Percentage CPU" `
-metricWindowSize "PT15M" `
-metricOperator "GreaterThan" `
-metricThreshold 80.0 `
-alertAdmins $true `
-alertOther "AppAdmin@contoso.com"