Create customized VM image for HPC Pack IaaS compute node

Article
08/31/2022

This article describes how you can create your own customized VM image to deploy IaaS compute nodes for your HPC Pack 2016 cluster.

There are two different ways to create the customized VM image, one for HPC Pack 2016 Update 3 and HPC Pack 2019, the other for HPC Pack 2016 RTM, HPC Pack 2016 Update 1 and HPC Pack 2016 Update 2. You shall select the correct way depending on the HPC Pack version of your head node(s).

Note

The difference between HPC Pack Update 3 VM image and HPC Pack Update 2 (and earlier) VM image is that HPC Pack compute node and Windows Management Framework 5.1 must be pre-installed in the latter.

Tip

You can also use Azure Image Builder feature to automate the customized VM image creation.

Create custom VM image for HPC Pack 2016 Update 3 and HPC Pack 2019

Step 1: Create a virtual machine in Azure

On Azure Portal, create an Azure virtual machine with one of the following operating systems: Windows Server 2012, Windows Server 2012 R2, Windows Server 2016, Windows Server 2019.

Step 2: Install and configure your own applications in the virtual machine

Log onto the virtual machine, and install and configure your own applications.

Step 3: Generalize the virtual machine using Sysprep

Detailed link: Generalize a VM before creating an image

Sign in to your Windows virtual machine.
Open a Command Prompt window as an administrator.
Delete the panther directory (C:\Windows\Panther).
Then change the directory to %windir%\system32\sysprep, and then run: sysprep.exe /oobe /generalize /mode:vm /shutdown

The VM will shut down when Sysprep is finished generalizing the VM. Do not restart the VM.

Once Sysprep has finished, set the status of the virtual machine to Generalized.

Set-AzVm -ResourceGroupName $rgName -Name $vmName -Generalized

Step 4: Create a VM image

Open the Azure Portal, find the virtual machine from the virtual machine list. On the Overview page of the virtual machine, click Capture to create a managed image from the virtual machine.

Open the Azure Portal.
In the menu on the left, click Virtual Machines and find the virtual machine.
On the Overview page for the virtual machine, on the upper menu, click Capture.
In Name, type the name that you would like to use for the image.
In Resource group either select Create new and type in a name, or select Use existing and select a resource group to use from the drop-down list.
Select Automatically delete this virtual machine after creating the image.
Click Create to create the VM image.

Create custom VM image for HPC Pack 2016 Update 2 (or earlier)

Step 1: Create or select an Azure IaaS compute node with HPC Pack 2016 Update 2 (or earlier version)

You can either create an Azure virtual machine and manually install the correct version of HPC Pack 2016 compute node on it, or select an currently running Azure IaaS compute node in your existing HPC Pack 2016 cluster.

If Windows Management Framework (WMF) 5.1 is not installed in the IaaS compute node, download and install it.

Step 2: Install and configure your own applications in the Azure IaaS compute node

Log onto the virtual machine, and install and configure your own applications.

Step 3: Uninstall the VM extension(s) from the virtual machine

On Azure Portal, find the corresponding Azure virtual machine, click Extensions, uninstall all the VM extensions if there is any.

Step 4: Clear the cluster specific information from compute node

Log onto the Azure IaaS compute node, open a Windows PowerShell console as administrator, and run the following PowerShell script to clear the cluster specific information.

# Step 1. Stop all the HPC services and change StartupType to Disabled
$hpcServices = @("HpcManagement", "HpcNodeManager", "msmpi", "HpcMonitoringClient", "HpcSoaDiagMon")
foreach($svc in $hpcServices)
{
    Stop-Service -Name $svc -Force -ErrorAction SilentlyContinue
    Set-Service -Name $svc -StartupType Disabled
}

# Step 2: Remove HPC logs and temp files from this node
$datadir = [System.Environment]::GetEnvironmentVariable("CCP_DATA", [System.EnvironmentVariableTarget]::Machine)
$logFilesPath = [System.IO.Path]::Combine($datadir, "LogFiles")
if(Test-Path $logFilesPath -PathType Container)
{
    Get-ChildItem -Path $logFilesPath | select -ExpandProperty FullName | Remove-Item -Force -Recurse -ErrorAction SilentlyContinue
}
Get-ChildItem -Path C:\Windows\Temp  | select -ExpandProperty FullName | Remove-Item -Force -Recurse -ErrorAction SilentlyContinue
Get-ChildItem -Path $env:TEMP | select -ExpandProperty FullName | Remove-Item -Force -Recurse -ErrorAction SilentlyContinue

# Step 3: Remove CustomData.bin if it exists
Remove-Item "$env:SYSTEMDRIVE\AzureData\CustomData.bin" -Force -ErrorAction SilentlyContinue

# Step 4: Remove the HPC communication certificate
$thumbprint = Get-ItemPropertyValue -Path HKLM:\SOFTWARE\Microsoft\HPC -Name SSLThumbPrint
Remove-Item Cert:\LocalMachine\My\$thumbprint -ErrorAction SilentlyContinue
Remove-Item Cert:\LocalMachine\Root\$thumbprint -ErrorAction SilentlyContinue

# Step 5: Clear the information of the current HPC cluster
Set-ItemProperty -Path HKLM:\SOFTWARE\Microsoft\HPC -Name ClusterConnectionString -Value ""
Set-ItemProperty -Path HKLM:\SOFTWARE\Microsoft\HPC -Name SSLThumbPrint -Value ""
Set-ItemProperty -Path HKLM:\SOFTWARE\Microsoft\HPC -Name NonDomainRole -Value 0 -Type DWord
if(Test-Path HKLM:\SOFTWARE\Wow6432Node\Microsoft\HPC)
{
    Set-ItemProperty -Path HKLM:\SOFTWARE\Wow6432Node\Microsoft\HPC -Name ClusterConnectionString -Value ""
    Set-ItemProperty -Path HKLM:\SOFTWARE\Wow6432Node\Microsoft\HPC -Name SSLThumbPrint -Value ""
    Set-ItemProperty -Path HKLM:\SOFTWARE\Wow6432Node\Microsoft\HPC -Name NonDomainRole -Value 0 -Type DWord
}

[Environment]::SetEnvironmentVariable("CCP_SCHEDULER", "", [System.EnvironmentVariableTarget]::Machine)
[Environment]::SetEnvironmentVariable("CCP_CLUSTERCONNECTIONSTRING", "", [System.EnvironmentVariableTarget]::Machine)

Step 5: Generalize the virtual machine using Sysprep