How to: Associate an Automated Test with a Test Case

You might begin your testing cycle by creating a manual test case that you decide later is a good test to automate. You want to be able to continue to run that test as part of a test plan. Or you might have a manual test case that you used to create an action recording when you ran your test. Now you have created a coded UI test from that action recording and you want to associate this coded UI test with the original test case and run it as an automated test. For more information about action recordings and coded UI tests, see Recording and Playing Back Manual Tests and Testing the User Interface with Automated UI Tests.

You associate an automated test with a test case using Visual Studio. After you create your automated test, you must check in your test project that contains the automated test and make sure that the test project is part of your build definition. Then you associate the automated test with a test case that has been added to a test plan that uses this build. For more information about checking in your test project and team build, see Add Files to Version Control and Building the Application.

Important

If you are not using Team Foundation Build to build your application and tests, you can still run automated tests using Microsoft Test Manager. You must create a build definition that just has a share location added that is where your assemblies for your tests are located.

Also, you might have created a set of automated tests that you want to run as part of a test plan using Microsoft Test Manager. You can import these automated tests from an assembly and create test cases associated with each test using a command line tool for Microsoft Test Manager. For more information about how to import automated tests, see How to: Create Test Cases from an Assembly of Automated Tests Using tcm.exe.

You can only associate the automated test with your test case using Visual Studio. You cannot make this association using Microsoft Test Manager. If you have an existing test case that you want to use, you must first open the test case using Visual Studio as shown in the following illustration.

Then you can associate the test method with your test case as shown in the following illustration.

If you need to create a new test case, then you can do this from the Test View window as described in the following procedure Create a Test Case From an Automated test. You must then add this test case to a test plan to be able to run it.

Note

You can view the information from Microsoft Test Manager, but you cannot modify it.

The following table shows the types of automated tests that you can create and which types of tests can be run as part of a test plan:

Type of test	Description	Run from a test plan by associating a test method with a test case
Coded UI Tests	Tests the user interface by performing UI actions.	Yes
Unit Tests	Tests code at the method level.	Yes
Database Unit Tests	Tests a stored procedure, function or trigger in a database.	Yes
Load tests	Tests application performance and stress using unit tests, Web Performance tests or coded UI tests.	Not recommended because you cannot view the test as it runs.
Web Performance Tests	Used as part of load tests to test server responses for Web applications by sending and receiving http requests. Note These tests cannot be used to test the UI.	Not recommended because Web Performance tests are primarily intended to be used with load tests.
Generic Tests	Test functionality using API calls or command line tools for the application under test.	Yes

Use the following procedures to associate an automated test with a test case that you can then run as part of your test plan:

• Add the automated test to your existing test case

• Create a test case from an automated test

• Set up your test plan to use your build

• Create your test settings and environment to run your tests

• Run the automated test using Microsoft Test Manager

• View and update the test results

Add the Automated Test to Your Test Case

The parameters in a test case are not used by any automated test that you associate with a test case. Iterations of a test case that use these parameters are for manual tests only.

To add the automated test to your test case

1. To open the solution that contains the automated test, click File and point to Open and then click Project/Solution.

   The Open Project dialog box is displayed.

2. Select your solution, and then click Open.

   Note

   You must open your solution to be able to associate the automation with the test case.

3. If you know the work item of the test case, in Team Explorer, on the Team menu, click Go To Work Item.

   You can also right-click the Work Items node for your team project.

4. In the Go to Work Item dialog box, in the ID box, type the ID number of your test case.

5. Click OK to open the work item.

   Note

   You can also use a pre-existing query in the Team Queries or My Queries node to find the work item if you do not know the ID. For more information about how to use queries in Team Explorer, see Finding Bugs, Tasks, and Other Work Items.

6. To add the automated test, click the Associated Automation tab.

7. To find the automated test name, click the ellipsis (…).

   The Choose Test dialog box is displayed. All the tests in the solution are shown in the list together with their associated test projects.

   Note

   If a test case already has an automated test associated with it, you must first remove this association before you can add a different automated test. Click Remove association to remove the existing automation.

8. Select the test, and then click OK.

   Note

   The value in Automation Status is automatically changed to Automated.

9. To save the changes to the test case, click Save Work Item.

   The next part of the process is to set up your test plan to use your build.

Create a Test Case from an Automated Test

To add the automated test to your test case

1. To open the solution that contains the automated test, click File and point to Open and then click Project/Solution.

   The Open Project dialog box is displayed.

2. Select your solution, and then click Open.

   Note

   You must open your solution to be able to associate the automation with the test case.

3. To open the Test View window, click Test, point to Windows, and then point to Test View.

   The Test View window is displayed.

4. To create a new test case for the automated test, right-click the test method in the Test View window and point to Create Test Case from Test.

   The fields in the Associated Automation tab are automatically filled in for the test method that you selected in the Test View window.

   Note

   The value in Automation Status is automatically changed to Automated.

5. Type the name of the test case in Title.

6. To set the priority of the test case, click Priority.

   Note

   For more information about the fields in a test case, see How to: Create a Manual Test Case.

7. To save the changes to the test case, click Save Work Item.

   You must add the test case to your test plan using Microsoft Test Manager. For more information, see How to: Add a Test Case to a Test Suite.

   The next part of the process is to set up your test plan to use your build.

Set Up Your Test Plan to Use Your Team Build

To set up your test plan to run the automated test that you have created, you must select the correct build definition used to build your automated test or a build definition that has the correct build drop location for your existing automated test assemblies. You must do this so that the automated test can be found in the share location for your build definition and then it can be run from Microsoft Test Manager.

Important

If you have multiple build configurations, the test assemblies to run the automated tests are searched for recursively from the root directory of the build drop folder. If it is important which assemblies are selected when you run your automated tests, you should use Run with options to specify the build configuration. For more information about how to do this, see How to: Override Settings in Your Test Plan For Test Runs.

To set up your test plan to use your team build

1. Open Microsoft Test Manager.

   Note

   To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010.

2. To select a test plan, click the down-arrow on the center group switcher and then click Testing Center.

3. On the center group menu bar, click Plan.

4. To set up your test plan to run the automated tests, click Properties and then click the drop-down arrow to the right of Filter for builds.

   The dialog box that shows build definition and quality is displayed.

5. To select the build definition that is used to build your automated tests, click Build definition.

6. Each build can be given a specific value to reflect the quality of the build. To select the quality of the builds you want to be able to view, click Build quality.

   Note

   For more information about build definitions and build quality, see Building the Application.

7. To save your changes, click Set build filter.

8. To select the most recent build to use with this test plan that includes the latest changes to the automated test, you must first click Save to save the plan and then click Modify.

   The Assign Build activity is displayed. You can compare your current build with a build you plan to take. The associated items list shows the changes to work items between the builds. You can then assign the latest build to take and use for testing with this plan. For more information about how to select a build, see Determining Which Builds Have Bug Fixes, New Features, or Requirements.

9. To close the Assign Build activity and return to the test plan properties, click the Close icon.

10. To save these changes for this test plan, click Save in the toolbar.

Create Your Test Settings and Environment to Run Your Tests

To run your automated tests, you must use a physical or virtual environment. You cannot run automated tests using Microsoft Test Manager without an environment.

You must create an environment that contains the roles in your test settings and then use this environment in your test plan. For more information about how to create your environment and roles and test settings, see Setting Up Test Machines to Run Tests or Collect Data.

Note

If you want to run automated tests that interact with the desktop, you must set up your agent to run as a process instead of a service. For more information, see How to: Set Up Your Test Agent to Run Tests that Interact with the Desktop.

To create your test settings and environment to run your tests

1. To create a physical environment, follow the steps in this topic: Creating a Physical Environment to Use for Testing.

2. If you are using Visual Studio Lab Management, you can create a virtual environment. To create a virtual environment, follow the steps in this topic: How to: Create an Environment from Virtual Machines or Templates. If you want more information about how to create virtual environments, see Virtual Environments Concepts and Guidelines.

3. To create your test settings, follow the steps in this topic: Create Test Settings for Automated Tests as Part of a Test Plan.

Run the Automated Test Using Microsoft Test Manager

To run the automated test using Microsoft Test Manager

1. Open Microsoft Test Manager.

   Note

   To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010.

2. To run the automated test, click the down-arrow on the center group switcher and then click Testing Center.

3. On the center group menu bar, click Test.

4. (Optional) To override the build, the test settings or the environment to use for running the automated tests that you select in this test plan, right-click the test and then click Run with options. For example, if you want to run on a staging environment instead of your standard testing environment then you might select a different environment. From the Run options dialog box, you can change these settings, and then click Run to run the selected test.

   Note

   If you select a different environment, it must contain the same roles that you selected in the test settings that you use.

5. To run the automated test without changing any options, right-click the test and then click Run.

   The Analyze Test Runs activity is displayed. It shows the progress of the test run that contains this test.

   Note

   You can run multiple automated tests by selecting multiple tests, or you can select to run a whole suite of tests. To run a suite, right-click the test suite and then click Run.

View and Update the Test Results

To view and update the test results

1. Open Microsoft Test Manager.

   Note

   To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010.

2. To view the test results, click the down-arrow on the center group switcher and then click Testing Center.

3. On the center group menu bar, click Test and then click Analyze Test Runs.

   The Analyze Test Runs activity is displayed. It shows any test runs for this test plan.

4. Double-click a test run to open it and view the details.

   The test run details are displayed.

5. (Optional) To update the title of your test run to be more meaningful, type the new name in Title.

6. (Optional) If your test failed, you can update the reason for the failure. Click Resolution and select the reason for the failure from the list.

7. (Optional) To add comments to the test result, click the Comments icon. Type your comments and then click Save comments.

8. (Optional) To view the details of an individual test, double-click the test.

   The test result is displayed. It shows the details from the test run, the attachments for data collected for this test result, and the test results history for that test. You can close this view to return to the test run.

   Note

   If, from your analysis, you determine that there is a bug, you can create a bug from this view.

9. To save these changes for this test run, click Save in the toolbar.

See Also

Concepts

How to: Create Test Cases from an Assembly of Automated Tests Using tcm.exe

Other Resources

Creating Automated Tests