Parameterizing Tests to Run in Different Environments

Sean has a great post on running Web tests and load tests from the command line.

A hidden feature is that you can set parameter values from environment variables, which enables you to target different environments with the same test without editing the test.

You may develop your web tests and load tests in one test environment, and then need to re-run it against another environment. A common practice is to have test, preproduction, and production environments.

In Web tests, you can use the Parameterize Web Server command to put the server part of the URL in a context parameter.

Now my target server has been parameterized out to the WebServer1 context parameter. Depending on your test, you may also need to add additional parameters. For example, some Web applications send the URL in a query string. In VS 2010, use the search command to find all instances of the target server and replace it with a context parameter.

That’s great, but does that mean you need to edit the Web test to target a different environment? No!

There are two ways to override Web test context parameters:

1) override the value from an environment variable. An environment variable with the name Test. WebServer1 will override the value of the WebServer1 context parameter. In a similar way you can override variable values in a unit test.

2) create a context parameter with the same name in the load test. The load test parameter value will override the Web test value. Here I’ve created a run settings that targets the test environment. Notice that the run setting is configured to collect counters from the target web server, and it overrides the context parameter value to set the same web server.

Now I can create a second Run Settings in my load test to target the pre-production environment:

Now I have two run settings in my load test, each targeting a different environment.

Great, does that mean I have to edit my load test to target a different environment? No!

Here is the real hidden gem: you can override the default run settings in your load test by setting an environment variable from the command line:

Set Test.UseRunSetting=PreProdEnvironment
mstest /testcontainer:loadtest1.loadtest 

These features have been in the product since VS 2005.

Ed.