Configure Azure Load Testing for high-scale load

Article
03/03/2025

In this article, you learn how to configure your load test for high-scale with Azure Load Testing. Azure Load Testing abstracts the complexity of provisioning the infrastructure for simulating high-scale traffic. To scale out a load test, you can configure the number of parallel test engine instances. To achieve an optimal load distribution, you can monitor the test instance health metrics in the Azure Load Testing dashboard.

Prerequisites

An Azure account with an active subscription. If you don't have an Azure subscription, create a free account before you begin.
An existing Azure load testing resource. To create an Azure load testing resource, see the quickstart Create and run a load test.

Configure load parameters for a load test

To simulate user traffic for your application, you can configure the load pattern and the number of virtual users you want to simulate load for. By running the load test across many parallel test engine instances, Azure Load Testing can scale out the number of virtual users that simulate traffic to your application. The load pattern determines how the load is distributed over the duration of the load test. Examples of load patterns are linear, stepped, or spike load.

Depending on the type of load test, URL-based, JMeter-based or Locust-based, you have different options to configure the target load and the load pattern. The following table lists the differences between the test types.

Test type	Number of virtual users	Load pattern
URL-based (basic)	Specify the target number of virtual users in the load test configuration.	Linear load pattern, based on the ramp-up time and number of virtual users.
URL-based (advanced)	Specify the number of test engines and the number of virtual users per instance in the load test configuration.	Configure the load pattern (linear, step, spike).
JMeter-based	Specify the number of virtual users per instance in the test script. Specify the number of test engines in the load test configuration.	Configure the load pattern in the test script.
Locust-based	Specify the total number of users in the load test configuration, the locust configuration file, or the test script. Specify the number of test engines in the load test configuration.	Configure the load pattern in the test script.

Configure load parameters for URL-based tests

To specify the load parameters for a URL-based load test:

Azure portal
Azure Pipelines / GitHub Actions

In the Azure portal, go to your Azure Load Testing resource.
In the left navigation, select Tests to view all tests.
In the list, select your load test, and then select Edit.

Alternately, you can also edit the test configuration from the test details page. To do so, select Configure, and then select Test.
On the Basics page, make sure to select Enable advanced settings.
On the Edit test page, select the Load tab.

For URL-based tests, you can configure the number of parallel test engine instances and the load pattern.
Use the Engine instances slider control to update the number of parallel test engine instances. Alternately, enter the target value in the input box.
Select the Load pattern value from the list.

For each pattern, fill the corresponding configuration settings. The chart gives a visual representation of the load pattern and its configuration parameters.

For CI/CD workflows, you specify the load parameters for a URL-based load test in the requests JSON file, in the testSetup property.

Depending on the load pattern (loadType), you can specify different load parameters:

Load pattern (`loadType`)	Parameters
Linear	`virtualUsersPerEngine`, `durationInSeconds`, `rampUpTimeInSeconds`
Step	`virtualUsersPerEngine`, `durationInSeconds`, `rampUpTimeInSeconds`, `rampUpSteps`
Spike	`virtualUsersPerEngine`, `durationInSeconds`, `spikeMultiplier`, `spikeHoldTimeInSeconds`

In the load test configuration YAML file, make sure to set the testType property to URL and set the testPlan property to reference the requests JSON file.

The following code snippet shows an example requests JSON file for a URL-based load test. The testSetup specifies a linear load pattern that runs for 300 seconds, with a ramp-up time of 30 seconds and five virtual users per test engine.

{
    "version": "1.0",
    "scenarios": {
      "requestGroup1": {
        "requests": [
          {
            "requestName": "Request1",
            "requestType": "URL",
            "endpoint": "http://northwind.contoso.com",
            "queryParameters": [],
            "headers": {},
            "body": null,
            "method": "GET",
            "responseVariables": [
                {
                  "extractorType": "XPathExtractor",
                  "expression": "/note/body",
                  "variableName": "token"
                }
              ]
          },
          {
            "requestName": "Request2",
            "requestType": "CURL",
            "curlCommand": "curl --location '${domain}' --header 'Ocp-Apim-Subscription-Key: ${token}"
          }
        ],
        "csvDataSetConfigList": [
            {
              "fileName": "inputData.csv",
              "variableNames": "domain"
            }
          ]
      }
    },
    "testSetup": [
      {
        "virtualUsersPerEngine": 5,
        "durationInSeconds": 300,
        "loadType": "Linear",
        "scenario": "requestGroup1",
        "rampUpTimeInSeconds": 30
      }
    ]
}

Configure load parameters for JMeter-based tests

To specify the load parameters for a JMeter-based load test:

Azure portal
Azure Pipelines / GitHub Actions

In the Azure portal, go to your Azure Load Testing resource.
In the left navigation, select Tests to view all tests.
In the list, select your load test, and then select Edit.

Alternately, you can also edit the test configuration from the test details page. To do so, select Configure, and then select Test.
On the Edit test page, select the Load tab. Use the Engine instances slider control to update the number of test engine instances, or enter the value directly in the input box.
Select Apply to modify the test and use the new configuration when you rerun it.

For CI/CD workflows, you configure the number of engine instances in the YAML test configuration file. You store the load test configuration file alongside the JMeter test script file in the source control repository.

Open the YAML test configuration file for your load test in your editor of choice.
Configure the number of test engine instances in the engineInstances setting.

The following example configures a load test that runs across 10 parallel test engine instances.
```
version: v0.1
testId: SampleTestCICD
displayName: Sample test from CI/CD
testPlan: SampleTest.jmx
description: Load test website home page
engineInstances: 10
```
Save the YAML configuration file, and commit the changes to source control.

Configure load parameters for Locust-based tests

To specify the load parameters for a Locust-based load test:

Azure portal
Azure Pipelines / GitHub Actions

In the Azure portal, go to your Azure Load Testing resource.
In the left navigation, select Tests to view all tests.
In the list, select your load test, and then select Edit.

Alternately, you can also edit the test configuration from the test details page. To do so, select Configure, and then select Test.
On the Edit test page, select the Load tab. Enter the values for the overall users required and overall spawn rate in the respective input boxes. The engine instance count required to generate this load automatically populates. If your test script is complex and resource intensive, use the Engine instances slider control to update the number of test engine instances, or enter the value directly in the input box.

Alternatively, you can configure the number of users and spawn rate in the test script or Locust configuration file and provide the number of engine instances required.

Select Apply to modify the test and use the new configuration when you rerun it.

For CI/CD workflows, you configure the number of engine instances in the YAML test configuration file. You can configure the overall users and spawn rate as environment variables in the YAML file. Alternatively, you can configure those in the test script or Locust configuration file. You store the load test configuration file alongside the Locust test script file in the source control repository.

Open the YAML test configuration file for your load test in your editor of choice.

Configure the number of test engine instances in the engineInstances setting. Configure the overall users and spawn rate as environment variables.

The following example configures a load test that runs across 10 parallel test engine instances.

version: v0.1
testId: SampleTestCICD
displayName: Sample test from CI/CD
testPlan: SampleTest.jmx
description: Load test website home page
engineInstances: 4
env:
  - name: LOCUST_USERS
    value: 2000
  - name: LOCUST_SPAWN_RATE
    value: 100
  - name: LOCUST_RUN_TIME
    value: 20

Save the YAML configuration file, and commit the changes to source control.

Monitor engine instance metrics

To make sure that the test engine instances, themselves aren't a performance bottleneck, you can monitor resource metrics of the test engine instance. A high resource usage for a test instance might negatively influence the results of the load test.

Azure Load Testing reports four resource metrics for each instance:

CPU percentage.
Memory percentage.
Network bytes per second.
Number of virtual users.

A test engine instance is considered healthy if the average CPU percentage or memory percentage over the duration of the test run remains below 75%.

To view the engine resource metrics:

Go to your Load Testing resource. On the left pane, select Tests to view the list of load tests.
In the list, select your load test to view the list of test runs.
In the test run list, select your test run.
In the test run dashboard, select the Engine health to view the engine resource metrics.

Optionally, select a specific test engine instance by using the filters controls.

Troubleshoot unhealthy engine instances

If one or multiple instances show a high resource usage, it could affect the test results. To resolve the issue, try one or more of the following steps:

Reduce the number of threads (virtual users) per test engine. To achieve a target number of virtual users, you might increase the number of engine instances for the load test.
Ensure that your script is effective, with no redundant code.
If the engine health status is unknown, rerun the test.

Determine requests per second

The maximum number of requests per second (RPS) that Azure Load Testing can generate for your load test depends on the application's latency and the number of virtual users (VUs). Application latency is the total time from sending an application request by the test engine, to receiving the response. The virtual user count is the number of parallel requests that Azure Load Testing performs at a given time.

To calculate the number of requests per second, apply the following formula: RPS = (# of VUs) * (1/latency in seconds).

For example, if application latency is 20 milliseconds (0.02 seconds), and you're generating a load of 2,000 VUs, you can achieve around 100,000 RPS (2000 * 1/0.02s).

To achieve a target number of requests per second, configure the total number of virtual users for your load test.

Note

Apache JMeter only reports requests that made it to the server and back, either successful or not. If Apache JMeter is unable to connect to your application, the actual number of requests per second will be lower than the maximum value. Possible causes might be that the server is too busy to handle the request, or that a TLS/SSL certificate is missing. To diagnose connection problems, you can check the Errors chart in the load testing dashboard and download the load test log files.

Test engine instances and virtual users for JMeter-based tests

In the Apache JMeter script, you can specify the number of parallel threads. Each thread represents a virtual user that accesses the application endpoint. We recommend that you keep the number of threads in a script below a maximum of 250.

In Azure Load Testing, test engine instances are responsible for running the Apache JMeter script. All test engine instances run in parallel. You can configure the number of instances for a load test.

The total number of virtual users for a load test is then: VUs = (# threads) * (# test engine instances).

To simulate a target number of virtual users, you can configure the parallel threads in the JMeter script, and the engine instances for the load test accordingly. Monitor the test engine metrics to optimize the number of instances.

For example, to simulate 1,000 virtual users, set the number of threads in the Apache JMeter script to 250. Then configure the load test with four test engine instances (that is, 4 x 250 threads).

The location of the Azure Load Testing resource determines the location of the test engine instances. All test engine instances within a Load Testing resource are hosted in the same Azure region.

Test engine instances and virtual users for Locust-based tests

Configure the total number of users required for your load test. This represents the peak number of concurrent Locust users. You can configure this in the load configuration while creating a test in Azure Load Testing. You can also configure this in your test script or the Locust configuration file. We recommend running up to 500 users from a test engine instance.

In Azure Load Testing, test engine instances are responsible for running the Locust script. All test engine instances run in parallel. You can configure the number of instances for a load test.

The total number of engines required for a load test is then: number of engines = (total number of users) / 500.

If you configure the total users in load configuration, Azure Load Testing populates the engine count automatically. If needed, you can update the number of engines. Monitor the test engine metrics to optimize the number of instances.

For example, to simulate 2,000 users, set the Total number of users in load configuration to 2,000. Azure Load Testing automatically populates the engine count as 4. Update the engine count if required. If you configured the users in your test script or Locust configuration file as 2,000, set the engine instances to 4 in the load configuration.