Automatically execute Coded UI test on different Lab VMs

Imagine this scenario: you are working in setting up the Lab Management features of TFS 2010. The main purpose is to execute automated tests. The customer asked to have a lab environment able to test different clients OS, namely Windows XP and Windows 7, each equipped with a number of browsers.

Sure, you could have multiple environments each with a single but different client, but this will require much more resources; the environment has already three servers: domain controller, SQL Server and a Web server. Two environments needs 2*(3+1)=8 VMs, while collapsing in one you have 3+2=5 VMs.

You are using the end-to-end workflow, i.e. LabDefaultTemplate.xaml, to deploy the application and test it. The template wizard has two combo boxes: one for Test Configuration and one for the Test Settings.

So you define a test setting, using the proposed defaults.

But then you realize that both client VMs are in the Web Client role: tests will be distributed among them!

The solution is to:

• using different roles for the two clients
• create different Test Configurations for each client role

As you see below the role are editable

 

In my case I gave distinct names to the client VMs, say “Web Client XP” and “Web Client 7”; then defined two Test Settings: in each setting, I selected a distinct VM where to run the test (see below picture).

Finally, I define two builds based on the E2E template, each pointing to the proper (automated) test settings, and linked to the matching Test Configuration (see below).

So far so good, but having two build definitions is a bit inelegant. Later I found this blog post https://blogs.msdn.com/b/lab_management/archive/2011/05/26/configuration-matrix-testing-using-visual-studio-lab-management.aspx, that suggest how to add a second test run with different settings to the E2E workflow.

Happy building!