Retirement guide for burst to Cloud Services with Microsoft HPC Pack
Microsoft High-Performance computing (HPC) Pack has a feature named Bursting to Cloud Service, including Azure Cloud Service worker role instances as compute node resource in the cluster. HPC Pack manages and monitors the communications with an Azure cloud service and storage account in your Azure subscription and gives you access to additional Azure features for your deployments. It allows you to create and manage Azure worker role compute instances besides dedicated on-premises Window or Linux compute nodes, part-time servers, workstation computers. You can use HPC Pack as your scheduler and have your jobs to run across all the compute resources.
Azure has announced the retirement of Cloud Service (classic) deployment model on 31 August 2024. HPC Pack product relies on cloud services configuration for the Bursting to Cloud Service feature. We will retire HPC Pack burst to Cloud Service on 31 August 2024. After retirement, you may not be able to create or manage Azure Cloud service worker role instances with HPC Pack as this feature will be removed from the product. It is strongly recommended that you stop using this feature and migrate the workload to Burst to Azure IaaS VM or Burst to Azure Batch service before the retirement date to keep workload continuity.
Migration options
There is no direct replacement for this feature. If you have Batch/MPI/SOA/Excel workloads and have ExpressRoute or Site to Site VPN, please consider migrating to Burst to IaaS VM. If you have a Batch/MPI workload and have only internet connection, please consider migrating to Burst to Azure Batch.
Another alternative is to deploy and manage the whole HPC Pack cluster in Azure instead of the three hybrid bursting solutions. This scenario is very similar to Burst to Azure IaaS VM, but having the head nodes, SQL servers and Domain Controllers in Azure together with the compute resources. You can use ARM templates to automatically deploy the whole cluster in Azure in tens of minutes.
Migration steps
Stop the workload and the Azure worker role instances
Select HPC Cluster Manager GUI.
Select Job Management, and select the active jobs from the list to cancel them.
If the Azure nodes are configured with automatic start and stop, go to Configuration and select Node Templates.
Open the Azure node template to change it to manual start and stop.
Select Resource Management and select all the Azure nodes and stop them.
Confirm that the nodes status have changed to state.
Delete the nodes with status Not-Deployed from the cluster.
Select Configuration. Then select the Node Templates.
Delete the Azure node templates.
Create new node templates for Burst to Azure IaaS VM or Burst to Azure Batch and start the Azure deployments for the nodes.
Refer to Burst to Azure IaaS VM from an HPC Pack Cluster for how to create the Azure IaaS node template and start the nodes.
Refer to Burst to Azure Batch with Microsoft HPC Pack for how to create the Azure Batch node template and start the nodes
Migrate the workloads
Burst to Azure IaaS VM is quite similar to on premise compute nodes with regard to the node communication, user identity and application management. Batch/MPI/SOA/Excel workloads can run the same way as running on on premise compute nodes.
Burst to Azure Batch service is using Azure Batch pool as compute node so it depends on features provided by Azure Batch. Currently only Batch/MPI workloads are supported to run on Batch pool node.
FAQ
Can Burst to IaaS VM or Burst to Azure Batch service support Linux nodes? Yes. Both Burst to IaaS VM and Burst to Azure Batch service support Linux nodes beside Windows nodes. This is different from Burst to Cloud Service which only Windows nodes are supported.
Can SOA/Excel workload run on Burst to IaaS VM or Burst to Azure Batch? SOA/Excel workload can only run on Burst to IaaS VM with Windows nodes. Note the SOA service deployment method is the same as that for on premise compute nodes. There is no need to use hpcpack.exe to create/upload/manage the SOA service package as with Burst to Cloud Service.
Are auto grow shrink and availability policy available for Burst to IaaS VM or Burst to Azure Batch? Auto grow and shrink is available for Burst to IaaS VM. For Burst to Azure Batch, the Batch pool auto scale formula is utilized. There is no availability policy for both.