End-to-End IaaS Workload Provisioning in the Cloud with Azure Automation and PowerShell DSC ( Part 1 )

In prior articles, we've discussed Azure Automation, Azure VM Agent extensions and PowerShell Desired State Configuration (DSC) as individual tools and approaches for automated workload provisioning.  I'm frequently asked about how these tools can be used together to orchestrate end-to-end provisioning of cloud workloads, including fabric, OS and application components.

This article is Part 1 of a two-part series. 

In this article, I'll provide an example of leveraging Azure Automation runbooks and PowerShell Workflows in a concerted approach to fully automate all aspects of provisioning new IaaS workloads on the Microsoft Azure cloud platform.

In Part 2 of this series, I'll provide an example of a working PowerShell DSC Configuration that can be used with Azure Automation runbooks to customize the configuration of operating system and application components running inside each  VM as part of the provisioning process.

What, when and where?

We certainly don't have a lack of automation technologies these days! In fact, one could argue that we have so many different tools for automation, that sometimes it can be confusing to know which tools to leverage for a particular scenario. As I've been helping customers with automated provisioning sequences, here's an approach that I've found works well:

1. Orchestrate overall provisioning process with Azure Automation
    
2. Provision cloud fabric resources with PowerShell Workflow activities using Azure Automation Runbooks
    
3. Configure OS and Application payloads inside Azure VMs with PowerShell DSC
    
4. Publish PowerShell DSC Configuration to Azure Storage Account
    
5. Use Azure VM Agent extensions to apply PowerShell DSC Configuration

In the near future, Azure Resource Manager (ARM) will also be worthwhile including in your automation toolkit to manage cloud resources as a related group of dependencies. However, currently automation of ARM is limited to a subset of Azure PaaS resources.  Azure IaaS support is coming soon, so be sure to watch this space for updates!

In the next few sections of this article, I'll step through the details around steps #1 and #2 above.  In part 2 of this two-part series, I'll finish-up with coverage of steps #3, #4 and #5.

Prepare your Azure subscription

The example in this article assumes that you already have a few basic IaaS components provisioned in your Azure subscription with a consistent naming convention, including:

• Affinity group: xxxlabag01
• Storage account: xxxlabstor01
• Virtual network: xxxlabnet01
• Windows Server Active Directory VM: xxxlabad01

When provisioning these components, replace xxx with your unique initials to arrive at a consistent naming convention that provides globally unique names for the storage account and cloud service for the Windows Server Active Directory domain controller VM. If you need help provisioning these IaaS components, check out the following resources to help you get started:

• Step-by-Step: Auto-Provision a New Active Directory Domain in the Azure Cloud
• Early Expert: Become a "Hybrid Cloud" Early Expert!

Orchestrate overall provisioning process with Azure Automation

Azure Automation provides a reliable, scalable platform service for orchestrating long-running processes that potentially touch several services and resources. Automation sequences are built as runbooks that include PowerShell Workflow activities. By leveraging PowerShell Workflows, Azure Automation runbooks can be suspended, resumed and recover from unexpected occurences while preserving current state of completed activities within the runbook.

If you haven't yet setup Azure Automation in your Microsoft Azure subscription or need some help with the basics of creating runbooks, check out this step-by-step article to walk through the process for getting started ...

• Step-by-Step: Getting Started with Azure Automation

Once your Azure Automation account is setup and ready to go, let's create a new Azure Automation runbook for provisioning our new cloud application, which we'll call MyWebApp in this article.

workflow New-MyWebAppDeployment
{

}

As mentioned above, Azure Automation runbooks leverage PowerShell Workflows, which is why we're defining our new runbook with the workflow keyword above.

• Tip! For step-by-step guidance on the basics of creating new runbooks, be sure to reference the Getting Started article linked above for a quick primer.

In an effort to make this runbook reusable for a variety of provisioning scenarios, I'll define a set of parameter inputs at the top of the workflow. Azure Automation will prompt for user input for each defined parameter when manually starting this runbook, or you can supply these parameter values when programmatically invoking this runbook from another runbook or PowerShell script.

workflow New-MyWebAppDeployment
{

param(

[parameter(Mandatory=$true)]
[String]
$subscriptionName,

[parameter(Mandatory=$true)]
[String]
$deploymentPrefix,

[parameter(Mandatory=$true)]
[Int]
$numberOfVMs,

[parameter(Mandatory=$true)]
[String]
$vmInstanceSize,

[parameter(Mandatory=$true)]
[String]
$vmDomain,

[parameter(Mandatory=$true)]
[String]
$vmDomainNetBIOS

)

}

Let's also declare a set of variables near the top of our workflow for common values that will be used when provisioning virtual machines,

workflow New-MyWebAppDeployment
{

       ... continued from above ...

       # Get Azure credential for authenticating to Azure subscriptions $cred = Get-AutomationPSCredential -Name 'AzureAutomationAccount'

   # Get vmAdmin credential for admin access inside each VM
$vmAdmin = Get-AutomationPSCredential -Name 'AzureAdmin'

# Set common variable values for provisioning each VM
$storageName = $deploymentPrefix + 'stor01'
$vmServiceName = $deploymentPrefix +'app'
$affinityGroupName = $deploymentPrefix + 'ag01'
$availabilitySetName = $deploymentPrefix + 'app'
$vNetName = $deploymentPrefix + 'net01'
$subnetName ='Subnet-1'
$dscArchive = 'AADSCWebConfig.ps1.zip'
$dscConfigName = 'WebSiteConfig'

}

Now that we've got the initial portion of our runbook created, let's save it as a draft and continue with adding the code that will provision our VM and application.

Provision cloud fabric resources with PowerShell Workflow activities

In the param section of our runbook workflow above, you'll note that we defined a $numberOfVMs parameter to accept, as input, the number of load-balanced VMs that we'd like to provision for our application.  To provide flexibility in provisioning a variable number of VMs, we'll add a for loop in our workflow that will handle the provisioning for each VM.

workflow New-MyWebAppDeployment
{

      ... continued from above ...

   for ($i=1; $i -le $numberOfVMs; $i++)
{

   }

}

As each iteration of the for loop above begins, we'll add code to checkpoint current workflow state and connect to the desired Azure subscription.

workflow New-MyWebAppDeployment
{

       ... continued from above ...

   for ($i=1; $i -le $numberOfVMs; $i++)
{

        # Save current state of workflow
Checkpoint-Workflow

# Connect to Azure subscription
Add-AzureAccount -Credential $cred

        Select-AzureSubscription `
-SubscriptionName $subscriptionName

# Set current Azure Storage Account
Set-AzureSubscription `
-SubscriptionName $subscriptionName `
-CurrentStorageAccountName $storageName

   }

}

Cloud workload provisioning tasks could involve many different cloud resources and be relatively long-running sequences.  Taking the above approach of checkpointing workflow state and reconnecting to the desired Azure subscription with each iteration of our for loop will provide the ability for this runbook to gracefully resume in the event that the workflow was suspended or unexpectedly interrupted.

Next, let's add our code for provisioning each VM inside the for loop we've defined above. Since provisioning a new Azure VM can involve several individual activities to build the desired VM configuration, we'll group all of these related steps into a single InlineScript workflow activity.

workflow New-MyWebAppDeployment
{

       ... continued from above ...

   for ($i=1; $i -le $numberOfVMs; $i++)
{

        Checkpoint-Workflow

# Connect to Azure subscription
Add-AzureAccount -Credential $cred

        Select-AzureSubscription `
-SubscriptionName $subscriptionName

# Set current Azure Storage Account
Set-AzureSubscription `
-SubscriptionName $subscriptionName `
-CurrentStorageAccountName $storageName

        # Provision VM
InlineScript {

$currentVM = "{0:D2}" -f $Using:i
$vmName = $Using:deploymentPrefix + 'app' + $currentVM
$vmImage = @((Get-AzureVMImage |
Where-Object Label -like `
"Windows Server 2012 R2 Datacenter*").ImageName)[-1]
$vmAdmin = $Using:vmAdmin

# Specify VM name, image and size
$vm = New-AzureVMConfig `
-Name $vmName `
-ImageName $vmImage `
-InstanceSize $Using:vmInstanceSize

# Specify VM local admin and domain join creds
$vm = Add-AzureProvisioningConfig `
-VM $vm `
-WindowsDomain `
-AdminUserName $vmAdmin.Username `
-Password $vmAdmin.GetNetworkCredential().Password `
-JoinDomain $Using:vmDomain `
-Domain $Using:vmDomainNetBIOS `
-DomainUserName $vmAdmin.Username `
-DomainPassword `
$vmAdmin.GetNetworkCredential().Password

# Specify load-balanced firewall endpoint for HTTPS
$vm = Add-AzureEndpoint `
-VM $vm `
-Name 'WebHTTPS' `
-LBSetName 'LBWebHTTPS' `
-DefaultProbe `
-Protocol tcp `
-LocalPort 443 `
-PublicPort 443

# Specify VNet Subnet for VM
$vm = Set-AzureSubnet `
-VM $vm `
-SubnetNames $Using:subnetName

# Specify HA Availability Set for VM
$vm = Set-AzureAvailabilitySet `
-VM $vm `
-AvailabilitySetName $Using:availabilitySetName

# Provision new VM with specified configuration
New-AzureVM `
-VMs $vm `
-ServiceName $Using:vmServiceName `
-VnetName $Using:vNetName `
-AffinityGroup $Using:affinityGroupName `
-WaitForBoot

}

    }

}

At this point, you can save and test your new runbook.  When testing the runbook, you'll be prompted to input values for each parameter we defined. When prompted for a DeploymentPrefix, enter xxxlab, where xxx is your unique initials used at the top of this article. This prefix will be prepended to the Azure resource names used within the runbook to reference your unique naming convention.

After the runbook completes, you will have multiple VMs in the specified Azure subscription, storage account, affinity group and virtual network.  Assuming you have a Windows Server Active Directory domain controller VM running on that same virtual network, each new VM will also be joined as a member server to the specified Active Directory domain.

What's next?

In Part 2 of this series, we'll walk through wiring in PowerShell DSC to configure the operating system and applications inside the VM as part of the provisioning process ...