Import test results into TFS using Orchestrator

In the last post I talked about how to manually create a build record in TFS.  In this post I will talk about how to import the test results into TFS using TFS IP.  Just like we said last time, some teams have their proprietary test infrastructure, the authoring/execution/management of test cases are performed outside of TFS.  In order to put all information in one place to facilitate project tracking and reporting (or for other reasons), we may want to import test case results into TFS.  Powered by TFS IP, Orchestrator can do this in a straightforward way.

Firstly, in order to conform to the TFS/Test Manager object model, test case results have to be associated with Test Cases (a specialized “work item”) which are grouped into Test Suite which is created under a Test Plan in a team project.  So we need to do the following:

1. Creates test cases using either Visual Studio or Test Manager 2010.
2. Creates a test plan in the team project where test cases are stored using Test Manager 2010.
3. Creates one or more test suites under the test plan using Test Manager 2010.

Test Manager 2010 is quite intuitive to use, if you are not sure, refer to MSDN documentation.

In this example, I created “Yao Test Plan” which has one test suite called “Yao Test Suites”.  There are two test cases in the test suite.  Here is how it looks like in Test Manager 2010:

I want to import one test case result “test case 320598 passed” into TFS.  In the real world scenario, you will have much more, but the basic steps are the same.

We can query all the test plans in a team project using “Get Test Plan” activity, the output is the list of Test Plan IDs:

I am only interested in my own plan, so I filter on the test plan name:

Next I use “Create Test Run” to create a “Test Run” object and set Owner to “Zhenhua Yao” (Owner should not be empty).  Test Run is the collection of all test case results that were executed together with a build, having a Test Run object is the prerequisite of importing test case results.  Note that I use the test plan ID queried from the last activity, you can also leave it empty and fill in the “Test Plan Name” if the name is unique.  Build Number usually refers to the build record in TFS (can be created by running a team build or creating a manual build), but in practice you can use any string or even leave it empty if you really don’t care the build.  In this activity, you also need to specify either “test suites” and/or “test cases” (they cannot be both empty), the test cases should have a 1:1 mapping to the test case results to be imported.  If a test case is not in the Test Run, you cannot use “Add Test Case Result” activity to add it, if a test case is in the Test Run but you have not imported the result, the test run is considered as uncompleted yet.  In most cases, you will put a comma separate test suite IDs in this activity.

After the test run is created, we can use “Add Test Case Result” activity to import the result one by one by its ID:

Note that Owner field must be set.  Again, if you don’t want to specify the test run ID, you can specify its title if it’s unique.

Once all test case results have been imported, you can use “Query Test Runs” activity to see the updated Test Run as well as the statistics.  Here is the query result in the Runbook Tester:

The test run also shows up in the Test Manager 2010 as expected:

We can use it as any other Test Runs created by Test Manager 2010 itself:

For more information about these activities, you may refer to the documentation on codeplex or just try it by yourself.  If you have questions or any feature request, please let me know.