Setting Up Test Machines to Run Tests or Collect Data
Using Visual Studio, you can run your tests and also collect data and diagnostics when you run your tests. You use test settings to specify the data and diagnostics that you want to collect. You can even select diagnostic data adapters that affect the way that your test machine performs. For example, you might want to create a video recording of your desktop while you run your test, or collect system information about your Web server. Or you might want to emulate a slow network to impose a bottleneck on the system.
To run tests remotely on multiple machines, or collect data and diagnostics remotely you must use a test controller and test agents. The test controller runs as a service and assigns tests to a test agent to run. In addition it can tell the test agent what data or diagnostics need to be collected. You can manage the test controller and agents by using Visual Studio, or if you register the test controller with Team Foundation Server, then you can manage the controller and agents by using Microsoft Test Manager.
If you have a distributed application, you define a role for each computer to use to run tests or collect data. For example, if you have an application that consists of a Web server, a database server, and a desktop client, you would define one role for each of these. The desktop client can run the tests and collect data locally, and the other roles can collect any data that you require on the machine that you assign to that role. You can also assign multiple machines to the same role.
If you are using Microsoft Test Manager, you create an environment for this set of roles. An environment is a collection of computers in which each computer has an assigned role.
The following sections of this topic provide more information about the ways to run tests and collect data, based on the type of tests that you run and whether you want to use an environment:
Manual Tests
Automated Tests
Environments
Manual Tests
It is recommended that you run your manual tests on a local machine that is not part of the environment. You can collect data or affect a test machine for your manual tests in the following ways:
Collect data on the local machine using default test settings
Collect data on a local machine specifying the data to collect
Collect data on local and remote tiers of your application
For more information about how to set up your test settings and environments for these situations, see Collect more diagnostic data in manual tests.
Automated Tests
You can run tests either by using Microsoft Test Manager or by using Visual Studio 2013.
If you plan to run your automated tests by using Microsoft Test Manager, you must use a lab environment that contains a set of roles to run your tests from your test plan. You must create a test controller that is registered with your team project in Team Foundation Server. However, Microsoft Test Manager will set up the test agent in each machine in the environment.
If you plan to run automated tests by using Visual Studio, you can just run your automated tests on your local machine and use test settings to collect data locally. If you want to collect data or affect the test machine for specific parts of a multitier application, you can select a test controller and test agents and add roles to use in your test settings. You should not register the test controller with Team Foundation Server. However, you must set up a test agent in each machine on which you plan to initiate tests or collect test data.
The following illustration shows a test controller and test agents that are installed on a machine for each role in an application under test and the tasks that the test agent can perform. The test controller manages the test agents that are registered to it.
Important
If you want to use a test controller as part of an environment by using Microsoft Test Manager, you must register it with Team Foundation Server, as shown in the following illustration. However, if you want to use a test controller from Visual Studio, do not register the test controller with Team Foundation Server.
.png "Test Controller and Test Agents")
Warning
The test agents and test controllers can be installed in different domains if your testing setup requires it.
Environments
If you use Microsoft Test Manager to conduct your tests, you create lab environments on which to run the tests. There are two kinds of environments: standard and SCVMM environments. A standard environment can use physical computers or virtual machines, and the virtual machines can run on any virtualization framework. An SCVMM environment uses only virtual machines that are managed by System Center Virtual Machine Manager (SCVMM).
Microsoft Test Manager can be used to set up both kinds of environment. In the case an SCVMM environment, you can stop and start environments, store environments in a library, and create multiple copies of them.
In both cases, you assign roles to each machine in the environment. For example, typical roles are Web Server and Desktop Client. The role names are used by your test workflow to determine what software and tests to deploy on each machine.
Test Manager inserts a test agent on each computer, which enables the test controller to deploy software, run tests, and collect test results.
For more information, see Test on a lab environment.
Tasks
Use the following topics to help set up your machines to run tests and collect data.
Tasks |
Associated topics |
|---|---|
Installing test controllers and test agents to use for running tests and collecting data: You can use test controllers in your test settings that you create by using Visual Studio. Alternatively, you can use test controllers with your environments. |
|
Setting up existing computers for running tests: A standard environment consists of test agents that are installed on physical machines or virtual machines. |
|
Setting up lab environments for running tests: A lab environment can include physical or virtual machines. |
|
Set up how to run tests or collect data by using test settings: You can create test settings and select the set of roles for your application under test and then select an environment that contains these roles in your test plan. You can also override these settings in your test plan for specific test runs. |