Exploratory Testing with Test Manager (Developer Preview)

A long time ago I remember a tester telling me that “one of the best tests of a system is when I’m not following a script per say but just ‘playing around’ with it.” I think that statement resonates to a lot of testers and with the recent release of the developer preview for Microsoft Test Manager (MTM); this type of testing is now possible.

Based on a test plan you now have the ability to link an exploratory test to an already established requirement (i.e. work item) or you can truly blaze a trail and start to conduct testing without this association.

I’ve chosen the latter to illustrate a few items. The first is to show just how quickly it is to create a test case based on using the system in an undefined manner and the second, attaching and linking a bug to said test case (if need be) in a matter of seconds. Let’s take an example to clarify this further. Let’s say I’m a tester working on the Fabrikam Fiber (a fictitious telecommunications company) Intranet portal and my persona is a service representative (SR). As a SR, I want to update a service ticket. The original test case passed but since a new version (with slightly different functionality) was recently deployed there appears to be a problem. I proceed to update the ticket and when I click the “Close” link an error occurs. How could we accomplish this same test with MTM and document it?

I’d start by using the Exploratory Testing feature and begin a test session. This is similar to an action recording but this time I’m not following any predefined steps but rather “winging it” when going through the system. Obviously this time I know the path to take because I want to duplicate an error I encountered earlier (as a tester) but this process holds true if I didn’t know where/when an error was occurring.

Once I come across an error I have the ability to open up a bug and a test case right then and there.

New bug:

New test case:

Having this capability eliminates the problem of “how did you get that error again?” that most develoers ask (and rightfully so). Along with the proper steps being captured for reproduction, all of the specifications for that error have been logged (i.e. system information [OS, processor, memory, screen, etc.]).

Sample system information XML file:

To confirm that the bug and test case were entered I can view my Exploratory Testing Sessions and see exactly what sessions have been completed. I also have the ability to pause and resume sessions if I get pulled away to work on another task. Notice the screen capture (along with the ID for the newly created bug and test case) which also helps document the error.

Combine all of the aforementioned items with video and audio and this leaves very little doubt to where and how this error was seen.

I foresee exploratory testing becoming an integral part of the overall test strategy as having the ability to really put the proposed system through its paces and not just going down the “happy path” will lead to more robust applications and higher code quality.

Comments

• Anonymous
  October 08, 2013
  Nice Article..!!Is there a way to export exploratory test session report to word or doc? I know about Test Scribe tool but seems like that doesn't support for exploratory session. Any idea?
• Anonymous
  October 09, 2013
  Vivek~ Unfortunately you can't export a session to a specific format so you'll have to copy-paste the findings into a report. That said, you can submit a feature request via: visualstudio.uservoice.com.