Edit

Share via

HPC system and big-compute solutions

Azure Batch
Azure Storage Accounts
Azure Virtual Machines

Solution ideas

This article describes a solution idea. Your cloud architect can use this guidance to help visualize the major components for a typical implementation of this architecture. Use this article as a starting point to design a well-architected solution that aligns with your workload's specific requirements.

This article describes a cloud-native application that uses Azure Batch. Batch provides compute resource allocation and management, application installation, resource autoscaling, and more.

Architecture

Architecture diagram that shows a cloud-native application that uses Azure Batch.

Download a Visio file of this architecture.

Dataflow

  1. Upload input files and the applications to your Azure Storage account.
  2. Create a Batch pool of compute nodes, a job to run the workload on the pool, and the tasks in the job.
  3. Batch downloads input files and applications.
  4. Batch monitors the task execution.
  5. Batch uploads the task output.
  6. Download the output files.

Components

Scenario details

Big compute and high performance computing (HPC) workloads are typically compute-intensive and can be run in parallel, taking advantage of the scale and flexibility of the cloud. The workloads are often run asynchronously using batch processing, with compute resources required to run the work and job scheduling required to specify the work.

This solution implements a cloud-native application with Azure Batch, which provides compute resource allocation and management, application installation, resource autoscaling, and job scheduling as a platform service. Batch also offers higher-level workload accelerators specifically for running R in parallel, AI training, and rendering workloads.

This solution is built on managed services including Virtual Machines, Storage, and Batch. These Azure services run in a high-availability environment, patched and supported, allowing you to focus on your solution.

Potential use cases

This solution is ideal for the finance, media, entertainment, energy, and environment industries. It's optimized for the following scenarios:

  • Financial risk Monte Carlo simulations (finance and portfolio)
  • Image rendering
  • Media transcoding
  • File processing
  • Engineering or scientific simulations (energy and environment)

Next steps

The following links provide documentation on deploying and managing the Azure products listed in the solution architecture: