Authoring with Azure Run As Certificate Credentials
As Azure becomes more prominent, I’m running into more asks on how monitor Azure from an on-premises perspective. As of this writing, we have a couple management packs available now that can help with this. One that monitors cloud services (VMs and storage), and it also discovers the presence of SQL Azure—System Center Management Pack for Windows Azure v1.1.238.0, 10/2/2014. And another for specifically monitoring SQL Azure databases—Windows Azure SQL Database Management Pack v1.5.4.0, 5/7/2013. There’s an older one out there as well, but that’s out of the scope of this post. Those are great, but they don’t cover everything as of yet. In the meantime, as per the guide for the Windows Azure MP says, any other gaps in monitoring need to be addressed by “[using] MP Authoring tools to create additional collection rules and monitors.” Sounds trivial enough…
When considering what’s involved with monitoring Azure, there are two major requirements: How to connect to Azure, and how to authenticate to Azure. The way the Windows Azure MP deals with the first is through a custom resource DLL included in the MPB file. The DLL contains functions that the MP leverages via Data Sources to feed the discoveries and monitoring workflows. While this is great for what’s provided and likely offers better performance at scale, it doesn’t offer much in terms of extensibility for Azure features outside the scope of the DLL. For any custom monitoring, I much rather rely on Azure PowerShell. If you’re already acquainted, great! If not, check it out, connect it to your subscription and poke around.
The first step in working with Azure PowerShell highlights the the other hurdle—authentication. Azure PowerShell typically uses a subscription registered with Add-AzureAccount, an interactive process that presents a windowed browser for logging in. Since interactive isn’t choice for automating or embedding in a terminal-less workflow, Set-AzureSubscription gives up a better method for automating the authentication to Azure using a management certificate. And so the sub-challenge of where to keep the management certificate presents itself. It is entirely possible to manually (or automatically for matter) deploy the certificate to the Personal Store, and then hardcode the private key in a custom management pack. Or, we can re-purpose…
The Windows Azure MP manages authentication through three Run As Profiles that it’s able to configure based on the responses entered into the Administration->Windows Azure->Add and Edit Subscription wizards. The three profiles are the certificate blob (a double-base64 encoded copy of the certificate), the certificate password (used to decrypt the private key), and the credentials to use with the proxy for the subscription. Or by their element IDs:
Microsoft.SystemCenter.WindowsAzure.RunAsProfile.Blob
Microsoft.SystemCenter.WindowsAzure.RunAsProfile.Password
Microsoft.SystemCenter.WindowsAzure.RunAsProfile.Proxy
It took a little reverse engineering to figure how to leverage these in a PowerShell workflow within SCOM, mainly due to trying to figure out the format of the certificate. Here’s the PowerShell from my SCOM discovery that I was able to come up with during testing:
# define certificate blob from run as profile
$CertBlob='$RunAs[Name="WindowsAzure!Microsoft.SystemCenter.WindowsAzure.RunAsProfile.Blob"]/Data$'
$CertPassword='$RunAs[Name="WindowsAzure!Microsoft.SystemCenter.WindowsAzure.RunAsProfile.Password"]/Password$'
$AzureSubId='$Target/Property[Type="WindowsAzure!Microsoft.SystemCenter.WindowsAzure.Subscription"]/SubscriptionId$'
$AzureSubName='$Target/Property[Type="System!System.Entity"]/DisplayName$'
# decode certificate blob from run as profile
$tempBytes=[System.Convert]::FromBase64String($CertBlob) # once
$tempString=[System.Text.Encoding]::Unicode.GetString($tempBytes)
$tempBytes=[System.Convert]::FromBase64String($tempString) # twice
$AzureCert=New-Object System.Security.Cryptography.X509Certificates.X509Certificate2 $AzureCert.Import($tempBytes,$CertPassword,[System.Security.Cryptography.X509Certificates.X509KeyStorageFlags]::MachineKeySet)
* Sorry for the word-wrap. I’ve done my best to ensure these copy out correctly.
And now there is a x509 certificate object $AzureCert that we can provide to Set-Azuresubscription:
Set-AzureSubscription -SubscriptionId $AzureSubId -Certificate $AzureCert -SubscriptionName $AzureSubId
Select-AzureSubscription -Current $AzureSubId
As for the proxy, Azure PowerShell uses the user or system’s proxy settings. To determine the proxy settings and user context the Windows Azure MP stores the proxy address as a property of the Azure Subscription and the Run As Profile associates to a Run As account that the data source will run as and authenticate (via negotiate presumably) to the proxy as. If no Run As Account is associated, the Default Action Account will be used. If the proxy doesn’t require authentication, then the below example will still work. Make sure to include this in any DataSourceType (WindowsAzure! reference assuming):
RunAs="WindowsAzure!Microsoft.SystemCenter.WindowsAzure.RunAsProfile.Proxy"
Then to use the proxy settings in PowerShell, thanks to several examples found on the Internet ultimately culminating to:
$webproxy = "$Target/Property[Type="WindowsAzure!Microsoft.SystemCenter.WindowsAzure.Subscription"]/ProxyServerAddress$"
if ($webproxy -notmatch "^https://") { $webproxy = "https://" + $webproxy }
$proxy = new-object System.Net.WebProxy
$proxy.Address = $webproxy
$proxy.Credentials = [System.Net.CredentialCache]::DefaultCredentials
[System.Net.WebRequest]::DefaultWebProxy = $proxy
And believe it or not Azure requests will be sent via the proxy. Next, we need some discovery data. I created a class name Azure.WebApp, and here’s the discovery bag script:
[string]$SourceId = "$Config/SourceId$"
[string]$ManagedEntityId="$Config/ManagedEntityId$"
$api = New-Object -ComObject ‘MOM.ScriptAPI’
$discoveryData = $api.CreateDiscoveryData(0, $SourceId, $ManagedEntityId)
foreach ($Website in Get-AzureWebsite) {
$Instance = $discoveryData.CreateClassInstance("$MPElement[Name='Azure.WebApp']$")
$Instance.AddProperty("$MPElement[Name='System!System.Entity']/DisplayName$", $Website.Name)
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/Enabled$", $Website.Enabled)
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/State$", $Website.State.ToString())
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/UsageState$", $Website.UsageState.ToString())
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/HostNames$", $Website.HostNames -join ';')
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/EnabledHostNames$", $Website.EnabledHostNames -join ';')
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/AdminEnabled$", $Website.AdminEnabled)
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/Sku$", $Website.Sku.ToString())
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/RepositorySiteName$", $Website.RepositorySiteName)
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/RepositoryUri$", $(($Website.SiteProperties.Properties | ? {$_.Name -eq "RepositoryUri"}).Value))
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/WebSpace$", $Website.WebSpace)
$discoveryData.AddInstance($Instance)
}
$api.Return($discoveryData)
#$discoveryData
Note the return—$api.Return($discoveryData) instead of the commented #$discoveryData. I use that because instead of using the “native” Microsoft.Windows.PowerShellDiscoveryProbe, I call powershell.exe with System.CommandExecuterProbePropertyBagBase instead (you’ll see several of the newer MPs doing this too). The reason to call out to powershell.exe instead of using the native module is that native module is using a PowerShell Workspace locked in at host version 2.0 and Azure PowerShell requires 3.0.
Altogether my current class type and data source type for the discovery looks like (note the hosting relationship with the subscription class):
<EntityTypes>
<ClassTypes>
<ClassType ID="Azure.WebApp" Accessibility="Public" Abstract="false" Base="Windows!Microsoft.Windows.ApplicationComponent" Hosted="true" Singleton="false" Extension="false">
<Property ID="Enabled" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="State" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="UsageState" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="HostNames" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="EnabledHostNames" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="AdminEnabled" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="Sku" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="RepositorySiteName" Type="string" AutoIncrement="false" Key="true" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="RepositoryUri" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="WebSpace" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="URL" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="NumberOfWorkers" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="WebSocketsEnabled" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="Use32BitWorkerProcess" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="DefaultDocuments" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="NetFrameworkVersion" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="PhpVersion" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="RequestTracingEnabled" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="HttpLoggingEnabled" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
<Property ID="DetailedErrorLoggingEnabled" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
</ClassType>
</ClassTypes>
<RelationshipTypes>
<RelationshipType ID="Microsoft.SystemCenter.WindowsAzure.Subscription.Hosts.AzureWebApp" Accessibility="Public" Abstract="false" Base="System!System.Hosting">
<Source ID="Source" MinCardinality="0" MaxCardinality="2147483647" Type="WindowsAzure!Microsoft.SystemCenter.WindowsAzure.Subscription" />
<Target ID="Target" MinCardinality="0" MaxCardinality="2147483647" Type="Azure.WebApp" />
</RelationshipType>
</RelationshipTypes>
</EntityTypes>
<ModuleTypes>
<DataSourceModuleType ID="Azure.WebApp.PowerShellDiscovery.DS" Accessibility="Public" Batching="false" RunAs="WindowsAzure!Microsoft.SystemCenter.WindowsAzure.RunAsProfile.Proxy">
<Configuration>
<xsd:element minOccurs="1" name="SourceId" type="xsd:string" xmlns:xsd="https://www.w3.org/2001/XMLSchema" />
<xsd:element minOccurs="1" name="ManagedEntityId" type="xsd:string" xmlns:xsd="https://www.w3.org/2001/XMLSchema" />
<xsd:element minOccurs="1" name="IntervalSeconds" type="xsd:integer" xmlns:xsd="https://www.w3.org/2001/XMLSchema" />
<xsd:element minOccurs="1" name="TimeoutSeconds" type="xsd:integer" xmlns:xsd="https://www.w3.org/2001/XMLSchema" />
<xsd:element minOccurs="1" name="PathToAzurePSModule" type="xsd:string" xmlns:xsd="https://www.w3.org/2001/XMLSchema" />
<xsd:element minOccurs="1" name="Debug" type="xsd:boolean" xmlns:xsd="https://www.w3.org/2001/XMLSchema" />
</Configuration>
<OverrideableParameters>
<OverrideableParameter ID="IntervalSeconds" Selector="$Config/IntervalSeconds$" ParameterType="int" />
<OverrideableParameter ID="TimeoutSeconds" Selector="$Config/TimeoutSeconds$" ParameterType="int" />
<OverrideableParameter ID="PathToAzurePSModule" Selector="$Config/PathToAzurePSModule$" ParameterType="string" />
<OverrideableParameter ID="Debug" Selector="$Config/Debug$" ParameterType="bool" />
</OverrideableParameters>
<ModuleImplementation Isolation="Any">
<Composite>
<MemberModules>
<DataSource ID="DS" TypeID="System!System.CommandExecuterDiscoveryDataSource">
<IntervalSeconds>$Config/IntervalSeconds$</IntervalSeconds>
<ApplicationName>%windir%\system32\WindowsPowerShell\v1.0\powershell.exe</ApplicationName>
<WorkingDirectory />
<CommandLine>-NoLogo -NoProfile -Noninteractive ". '$File/DiscoverAzureWebApps.ps1$'"</CommandLine>
<TimeoutSeconds>$Config/TimeoutSeconds$</TimeoutSeconds>
<RequireOutput>true</RequireOutput>
<Files>
<File>
<Name>DiscoverAzureWebApps.ps1</Name>
<Contents><![CDATA[
param(
[string]$SourceId = "$Config/SourceId$"
,[string]$ManagedEntityId="$Config/ManagedEntityId$"
,[string]$debug="$Config/Debug$"
)
# normalize debug to bool
if ($debug -eq "1" -or $debug.ToLower() -eq "true") {
$debug = $true
} else {
$debug = $false
}
# configure proxy
$webproxy = "$Target/Property[Type="WindowsAzure!Microsoft.SystemCenter.WindowsAzure.Subscription"]/ProxyServerAddress$"
If ($webproxy -ne "") {
if ($webproxy -notmatch "^https://") { $webproxy = "https://" + $webproxy }
if ($debug) { $api.LogScriptEvent("DiscoverAzureWebApps.ps1", 701, 0, "Using proxy: $webproxy") }
$error.Clear()
$proxy = new-object System.Net.WebProxy
$proxy.Address = $webproxy
#$account = new-object System.Net.NetworkCredential($user,[Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($pwd)), "")
#$proxy.credentials = $account
$proxy.Credentials = [System.Net.CredentialCache]::DefaultCredentials
[System.Net.WebRequest]::DefaultWebProxy = $proxy
if ($error.Count -gt 0) {
if ($debug) { $api.LogScriptEvent("DiscoverAzureWebApps.ps1", 703, 0, "Error configuring proxy: $error") }
}
}
# api and discovery bag staging
$api = New-Object -ComObject 'MOM.ScriptAPI'
$DiscoveryData = $api.CreateDiscoveryData(0, $SourceId, $ManagedEntityId)
# import module for Azure (hardcoded for now)
$error.Clear()
import-module '$Config/PathToAzurePSModule$'
if ($error.Count -gt 0) {
if ($debug) { $api.LogScriptEvent("DiscoverAzureWebApps.ps1", 704, 0, "Error importing Azure PowerShell module: $error") }
exit 1
}
# define certificate blob from run as profile
$CertBlob='$RunAs[Name="WindowsAzure!Microsoft.SystemCenter.WindowsAzure.RunAsProfile.Blob"]/Data$'
$CertPassword='$RunAs[Name="WindowsAzure!Microsoft.SystemCenter.WindowsAzure.RunAsProfile.Password"]/Password$'
$AzureSubId='$Target/Property[Type="WindowsAzure!Microsoft.SystemCenter.WindowsAzure.Subscription"]/SubscriptionId$'
$AzureSubName='$Target/Property[Type="System!System.Entity"]/DisplayName$'
# decode certificate blob from run as profile
$tempBytes=[System.Convert]::FromBase64String($CertBlob)
$tempString=[System.Text.Encoding]::Unicode.GetString($tempBytes)
$tempBytes=[System.Convert]::FromBase64String($tempString)
$AzureCert=New-Object System.Security.Cryptography.X509Certificates.X509Certificate2
$AzureCert.Import($tempBytes,$CertPassword,[System.Security.Cryptography.X509Certificates.X509KeyStorageFlags]::MachineKeySet)
if ($error.Count -gt 0) {
if ($debug) { $api.LogScriptEvent("DiscoverAzureWebApps.ps1", 704, 0, "Error reading Azure certificate: $error") }
exit 1
}
# add Azure subscription, name it after the ID to keep it simple and unique
$error.Clear()
Set-AzureSubscription -SubscriptionId $AzureSubId -Certificate $AzureCert -SubscriptionName $AzureSubId
if ($error.Count -gt 0) {
if ($debug) { $api.LogScriptEvent("DiscoverAzureWebApps.ps1", 704, 0, "Error setting Azure subscription: $error") }
exit 1
}
# select Azure subscripton as current
Select-AzureSubscription -Current $AzureSubId
$error.Clear()
# loop through web apps and create a discovered instance for each
foreach ($WebApp in Get-AzureWebsite) {
$Instance = $DiscoveryData.CreateClassInstance("$MPElement[Name='Azure.WebApp']$")
$Instance.AddProperty("$MPElement[Name='System!System.Entity']/DisplayName$", $WebApp.Name)
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/Enabled$", $WebApp.Enabled)
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/State$", $WebApp.State.ToString())
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/UsageState$", $WebApp.UsageState.ToString())
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/HostNames$", $WebApp.HostNames -join ';')
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/EnabledHostNames$", $WebApp.EnabledHostNames -join ';')
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/AdminEnabled$", $WebApp.AdminEnabled)
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/Sku$", $WebApp.Sku.ToString())
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/RepositorySiteName$", $WebApp.RepositorySiteName)
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/RepositoryUri$", $(($WebApp.SiteProperties.Properties | ? {$_.Name -eq "RepositoryUri"}).Value))
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/WebSpace$", $WebApp.WebSpace)
$Instance.AddProperty("$MPElement[Name='WindowsAzure!Microsoft.SystemCenter.WindowsAzure.Subscription']/SubscriptionId$", $AzureSubId)
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/NumberOfWorkers$", $WebApp.NumberOfWorkers)
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/WebSocketsEnabled$", $WebApp.WebSocketsEnabled)
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/Use32BitWorkerProcess$", $WebApp.Use32BitWorkerProcess)
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/DefaultDocuments$", $WebApp.DefaultDocuments -join ';')
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/NetFrameworkVersion$", $WebApp.NetFrameworkVersion)
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/PhpVersion$", $WebApp.PhpVersion)
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/RequestTracingEnabled$", $WebApp.RequestTracingEnabled)
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/HttpLoggingEnabled$", $WebApp.HttpLoggingEnabled)
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/DetailedErrorLoggingEnabled$", $WebApp.DetailedErrorLoggingEnabled)
if ($WebApp.HostNames.Count -gt 1) {
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/URL$", "https://$($WebApp.HostNames[0])")
} else {
$Instance.AddProperty("$MPElement[Name='Azure.WebApp']/URL$", "https://$($WebApp.HostNames)")
}
$DiscoveryData.AddInstance($Instance)
if ($debug) { $api.LogScriptEvent("DiscoverAzureWebApps.ps1", 700, 0, "Added Azure WebApp Instance: $($WebApp.Name)") }
}
if ($error.Count -gt 0) {
if ($debug) { $api.LogScriptEvent("DiscoverAzureWebApps.ps1", 704, 0, "Error reading Azure Web Apps: $error") }
exit 1
}
$api.Return($DiscoveryData)
]]></Contents>
</File>
</Files>
</DataSource>
</MemberModules>
<Composition>
<Node ID="DS" />
</Composition>
</Composite>
</ModuleImplementation>
<OutputType>System!System.Discovery.Data</OutputType>
</DataSourceModuleType>
</ModuleTypes>
Well that’s a start and should help lay some ground work for custom authoring of Azure monitoring. Next round I’ll finish this up with some monitors and talk about ideas around distributing and monitoring for the expected version of Azure PowerShell.