Use RDMA or GPU instances in Batch pools
To run certain Batch jobs, you can take advantage of Azure VM sizes designed for large-scale computation. For example:
To run multi-instance MPI workloads, choose HB, HC, NC, or ND series or other sizes that have a network interface for Remote Direct Memory Access (RDMA). These sizes connect to an InfiniBand network for inter-node communication, which can accelerate MPI applications.
For CUDA applications, choose N-series sizes that include NVIDIA Tesla graphics processing unit (GPU) cards.
This article provides guidance and examples to use some of Azure's specialized sizes in Batch pools. For specs and background, see:
Note
Certain VM sizes might not be available in the regions where you create your Batch accounts. To check that a size is available, see Products available by region and Choose a VM size for a Batch pool.
Dependencies
The RDMA or GPU capabilities of compute-intensive sizes in Batch are supported only in certain operating systems. The supported operating systems for these VM sizes include only a subset of those available for virtual machine creation. Depending on how you create your Batch pool, you might need to install or configure extra driver or other software on the nodes. The following tables summarize these dependencies. See linked articles for details. For options to configure Batch pools, see later in this article.
Linux pools - Virtual machine configuration
| Size | Capability | Operating systems | Required software | Pool settings |
|---|---|---|---|---|
| H16r, H16mr NC24r, NC24rs_v2, NC24rs_v3, ND24rs* |
RDMA | Ubuntu 22.04 LTS (Azure Marketplace) |
Intel MPI 5 Linux RDMA drivers |
Enable inter-node communication, disable concurrent task execution |
| NCv3, NDv2, NDv4, NDv5 series | NVIDIA Tesla GPU (varies by series) | Ubuntu 22.04 LTS (Azure Marketplace) |
NVIDIA CUDA or CUDA Toolkit drivers | N/A |
| NVv3, NVv4, NVv5 series | Accelerated Visualization GPU | Ubuntu 22.04 LTS (Azure Marketplace) |
NVIDIA GRID drivers (if required) | N/A |
*RDMA-capable N-series sizes also include NVIDIA Tesla GPUs
Important
This document references a release version of Linux that is nearing or at, End of Life(EOL). Please consider updating to a more current version.
Windows pools - Virtual Machine Configuration
| Size | Capability | Operating systems | Required software | Pool settings |
|---|---|---|---|---|
| H16r, H16mr NC24r, NC24rs_v2, NC24rs_v3, ND24rs* |
RDMA | Windows Server 2016, 2012 R2, or 2012 (Azure Marketplace) |
Microsoft MPI 2012 R2 or later, or Intel MPI 5 Windows RDMA drivers |
Enable inter-node communication, disable concurrent task execution |
| NC, NCv2, NCv3, ND, NDv2 series | NVIDIA Tesla GPU (varies by series) | Windows Server 2016 or 2012 R2 (Azure Marketplace) |
NVIDIA CUDA or CUDA Toolkit drivers | N/A |
| NV, NVv2, NVv4 series | NVIDIA Tesla M60 GPU | Windows Server 2016 or 2012 R2 (Azure Marketplace) |
NVIDIA GRID drivers | N/A |
*RDMA-capable N-series sizes also include NVIDIA Tesla GPUs
Windows pools - Cloud Services Configuration
Warning
Cloud Services Configuration pools are deprecated. Please use Virtual Machine Configuration pools instead.
| Size | Capability | Operating systems | Required software | Pool settings |
|---|---|---|---|---|
| H16r, H16mr | RDMA | Windows Server 2016, 2012 R2, 2012, or 2008 R2 (Guest OS family) |
Microsoft MPI 2012 R2 or later, or Intel MPI 5 Windows RDMA drivers |
Enable inter-node communication, disable concurrent task execution |
Note
N-series sizes are not supported in Cloud Services Configuration pools.
Pool configuration options
To configure a specialized VM size for your Batch pool, you have several options to install required software or drivers:
For pools in the virtual machine configuration, choose a preconfigured Azure Marketplace VM image that has drivers and software preinstalled. Examples:
Data Science Virtual Machine for Linux or Windows - includes NVIDIA CUDA drivers
Linux images for Batch container workloads that also include GPU and RDMA drivers:
Ubuntu Server (with GPU and RDMA drivers) for Azure Batch container pools
Create a custom Windows or Linux VM image with installed drivers, software, or other settings required for the VM size.
Create a Batch application package from a zipped driver or application installer. Then, configure Batch to deploy this package to pool nodes and install once when each node is created. For example, if the application package is an installer, create a start task command line to silently install the app on all pool nodes. Consider using an application package and a pool start task if your workload depends on a particular driver version.
Note
The start task must run with elevated (admin) permissions, and it must wait for success. Long-running tasks will increase the time to provision a Batch pool.
Example: NVIDIA GPU drivers on Windows NC VM pool
To run CUDA applications on a pool of Windows NC nodes, you need to install NVIDIA GPU drivers. The following sample steps use an application package to install the NVIDIA GPU drivers. You might choose this option if your workload depends on a specific GPU driver version.
- Download a setup package for the GPU drivers on Windows Server 2016 from the NVIDIA website - for example, version 411.82. Save the file locally using a short name like GPUDriverSetup.exe.
- Create a zip file of the package.
- Upload the package to your Batch account. For steps, see the application packages guidance. Specify an application ID such as GPUDriver, and a version such as 411.82.
- Using the Batch APIs or Azure portal, create a pool in the virtual machine configuration with the desired number of nodes and scale. The following table shows sample settings to install the NVIDIA GPU drivers silently using a start task:
| Setting | Value |
|---|---|
| Image Type | Marketplace (Linux/Windows) |
| Publisher | MicrosoftWindowsServer |
| Offer | WindowsServer |
| Sku | 2016-Datacenter |
| Node size | NC6 Standard |
| Application package references | GPUDriver, version 411.82 |
| Start task enabled | True Command line - cmd /c "%AZ_BATCH_APP_PACKAGE_GPUDriver#411.82%\\GPUDriverSetup.exe /s"User identity - Pool autouser, admin Wait for success - True |
Example: NVIDIA GPU drivers on a Linux NC VM pool
To run CUDA applications on a pool of Linux NC nodes, you need to install necessary NVIDIA Tesla GPU drivers from the CUDA Toolkit. The following sample steps create and deploy a custom Ubuntu 22.04 LTS image with the GPU drivers:
- Deploy an Azure NC-series VM running Ubuntu 22.04 LTS. For example, create the VM in the US South Central region.
- Add the NVIDIA GPU Drivers extension to the VM by using the Azure portal, a client computer that connects to the Azure subscription, or Azure Cloud Shell. Alternatively, follow the steps to connect to the VM and install CUDA drivers manually.
- Follow the steps to create an Azure Compute Gallery image for Batch.
- Create a Batch account in a region that supports NC VMs.
- Using the Batch APIs or Azure portal, create a pool using the custom image and with the desired number of nodes and scale. The following table shows sample pool settings for the image:
| Setting | Value |
|---|---|
| Image Type | Custom Image |
| Custom Image | Name of the image |
| Node agent SKU | batch.node.ubuntu 22.04 |
| Node size | NC6 Standard |
Example: Microsoft MPI on a Windows H16r VM pool
To run Windows MPI applications on a pool of Azure H16r VM nodes, you need to configure the HpcVmDrivers extension and install Microsoft MPI. Here are sample steps to deploy a custom Windows Server 2016 image with the necessary drivers and software:
- Deploy an Azure H16r VM running Windows Server 2016. For example, create the VM in the US West region.
- Add the HpcVmDrivers extension to the VM by running an Azure PowerShell command from a client computer that connects to your Azure subscription, or using Azure Cloud Shell.
- Make a Remote Desktop connection to the VM.
- Download the setup package (MSMpiSetup.exe) for the latest version of Microsoft MPI, and install Microsoft MPI.
- Follow the steps to create an Azure Compute Gallery image for Batch.
- Using the Batch APIs or Azure portal, create a pool using the Azure Compute Gallery and with the desired number of nodes and scale. The following table shows sample pool settings for the image:
| Setting | Value |
|---|---|
| Image Type | Custom Image |
| Custom Image | Name of the image |
| Node agent SKU | batch.node.windows amd64 |
| Node size | H16r Standard |
| Internode communication enabled | True |
| Max tasks per node | 1 |