Industrial Strength Exploratory Testing - part 2
In the last part, I introduced exploratory testing and looked at some advantages of employing exploratory testing. In this part, let’s look at a few myths surrounding exploratory testing and try to debunk some of these.
Myth 1: “Exploratory testing is ad-hoc, monkey testing”
Exploratory testing is NOT random, undirected testing. On the contrary, unlike scripted testing, you cannot just follow a set of pre-determined steps that might arrive at a defect. Testers often have hunches about where defects could be – a particular dev tends to be bad at exception handling, or an app that is weak in dealing with frequently disconnecting connections, or a website doesn’t do effective load balancing when stressed. As you learn more about the system under test, you get new hunches, new test ideas that could lead to unfound defects.
Vectoring exploratory testing with some of these test ideas will help explore the system more effectively and completely. Two great ways of vectoring exploratory testing can be found in session based test management by James Bach and exploratory testing tours by James Whittaker. SBTM will help direct exploratory testing by getting the tester to define a mission upfront and help keep exploration focused in that area. Testing tours help define various paths to explore in the system with each tour leading to different kinds of defects in different areas.
We use the concept of testing tours extensively in our in house testing – check out how our testing tours look:
Tours are essentially used to define a theme for exploration. If you were a tourist in a new city, how would you explore the city? Perhaps you would visit the famous landmarks in a quick tour, perhaps you would do a detailed walking tour of just one of your favorite sites, perhaps you would visit the lesser known attractions and avoid the touristy places. Likewise, if you were a tester testing a new app, how would you test it? Perhaps you would do a quick sales demo walking through the primary scenarios, perhaps you would test one scenario in great detail, perhaps you would test some of the lesser advertised features. Any theme that you choose to apply to your exploration is informed by the learning you have had from your previous explorations. You know your dev is lousy at exception handling - a tour like "Intelligent tourist tour" (where the tourist asks hard questions to the guide :D) would help - you could try all the hard boundary value inputs or rarer scenarios bound to produce a boundary condition. You know the app has perf issues when more than a few users hit the server - you could try the "Carnival tour" where your entire test team logs on to the client altogether and parties on it! :)
My favorite tour is the Back Alley tour where you exercise some of the lesser known paths in the application, the Fedex tour where you can track a data structure from point to point to see how it gets transmuted, the All Nighter tour that has caught some nasty memory leaks we fixed recently. More mythbusting in the next part...
Comments
- Anonymous
October 21, 2011
I like the tour names! That is something I will take away from this blog post :-] Good work! - Anonymous
November 07, 2011
Hi,I always liked explorery testing as tours. You should consider sharing your test suite (maybe with a test scribe generation). I think the details you would help the generalist testing community.[Anu] Yep - this is a great idea. I will run this by my test lead to figure how we can share this out. Thanks!