Test Impact Analysis

Let’s face it, when it comes to software development there’s always a bit of tension when a tester enters a bug and assigns it to the developer who worked on that piece of code. No one likes to have someone say: “Hey, you made a mistake” or “This is broke and it needs fixing.” Never the less, this is an important part of the process and really, when you think about it is essential to producing high quality applications.

While testers and developers may never see eye-to-eye (similar to the VB vs. C# crusades), having tools to eliminate bugs before they appear and capture the necessary data to reproduce them if they do will help immensely towards mediation on both sides of the fence.

Note: Test Impact Analysis is not supported on obfuscated binaries.

Test Impact View

Enter the “Test Impact View.” This view shows all tests that have been affected by recent code changes. After a baseline has been set (the initial build of the solution) subsequent builds with code changes will be compared and any tests that have methods affected will be shown.

The example I’m illustrating here is with the tried and true sample project for Tailspin Toys. I’ve made a change to the “AddItem” method within the “ShoppingCart” class (instead of adding 1 as the default quantity, 3 will be added).

A test project has been created and within it multiple tests for products, the shopping cart and other areas. Specifically in the shopping cart, there are 4 test methods that will be affected by this change after the next compilation.

Notice here that not only are the impacted tests displayed but also where the code was changed as well as any manual tests within TFS.

Running the impacted tests show that while 4 are impacted, only 2 fail so although there were changes to the code not all the tests were negatively impacted.

Examining the relatded test case within TFS further, opening the test case shows what steps are to be tested, associated user stories and other pertinent information to the test case (links, notes, etc.). This integration with TFS is certainly an added benefit as it brings visibility into the manual and automated testing process.

Obviously the next step would be to rectify the failed tests (in this case changing the code although that might not always be necessary) and re-run them so a fully functional code base can be built and deployed. This is also a good example of having TFS policies in place such as Gated Check-in’s (accept check-ins only if the submitted changes merge and build successfully) or enabling the “Builds” policy (requires that the last build was successful for each affected continuous integration build definition).

Summary

I should also make mention of IntelliTrace as that’s a fantastic feature for both developers and testers. For the former it can help tremendously during development thus eliminating time consuming diagnosis and the latter provides the QA team (and for that matter end users) with the ability to relay specific errors encountered (along with environment information) to the development staff. Justin Marks has a great in depth MSDN article on IntelliTrace so most definitely check it out.