Tourism and the art of Agile Testing

 This post is from an article I contributed to Quest Magazine. Check them out at https://www.qaiquest.org/2013/quest-magazine/?utm_source=QAI+Global+Institute&utm_campaign=e20dee456c-QUEST_Magazine_03.22.2013&utm_medium=email

Recently, I was fortunate to have the opportunity to speak at QAI TesTrek Toronto. I spoke on the tactics that a QA professional can use to work effectively with development teams who are adopting and working using agile techniques, like Scrum or Kanban.

One of the fundamental disconnects between a traditional waterfall approach to testing and these agile development methodologies is the dynamic nature of addressing the requirement backlog items for the project. This often leads to an unpredictable flow of requirements when viewed at project startup and can introduce planning challenges, as we don’t know at project inception when certain pieces of functionality will be ready to test. As such, we can’t accurately plan our testing effort end-to-end from the start of the project and instead must to adapt to rapid delivery iterations of functionality from the team. While this still makes it possible to plan by creating test cases that are linked to your requirements, the flow of delivery in functionality may not be predictable as in traditional waterfall execution. That being said, with thoughtful test authoring, you can still map out some of your test runs up front.

Modern, agile methodologies do present a core issue, however. With short development cycles, there is little time to write formal test scripts and, as features in the application tend to evolve rapidly, the formal test scripts that you do create can become outdated very quickly. You may find that you end up spending all of your time maintaining broken test scripts, rather than extending your test coverage.

Exploratory testing techniques can bring some relief from these issues. In exploratory testing, the testers are encouraged to plan as they test. They use their observations during testing to guide their future testing efforts, dynamically. As the flow of features comes from development, your examination of those features will lead you to a better understanding of how the application is performing, where it is lacking, and, overall, how upcoming features might impact the current state.

Exploratory testing can seem quite ad-hoc, however, with the intuition and experience of testers, it can be extremely powerful. Indeed, many successful testers make use of formally planned test scripts in addition to exploratory testing. In this case, the formal scripts guide the exploratory testing and the exploratory tests become variations on key scenarios, allowing far broader coverage. It introduces the ability to cover detours, if you will, to the expected or planned user paths through the application on which the formal scripts are usually based. I’ve had customers tell me that often the most significant bugs are often found using this approach.

When performing exploratory testing, either guided by a formal test plan or ad-hoc, there are even some interesting approaches to planning activities. A common practice is to make use of Tours.

Think about going to a large, busy city for the first time. There are tons of places to go and many different ways to get to those places. In addition, it becomes obvious that in a huge city like New York City, for example, it could take a very long time to cover all the places that one can see, particularly if you end up going to these places multiple times via different routes.

Software is similar to this. There are numerous aspects to test and even more paths to navigate between those aspects. There is just no practical way that a tester can be sure they’ve covered every feature of the software in every possible combination of paths.

With that in mind, why not take the approach of using exploratory testing and modeling your sessions on different types of tourism. For example, you might want to test the “main features” in the application, which will probably correlate to what end users will learn in their training or in the product documentation. Think of this like a tourist guidebook that highlights all the best places to stay and eat, the top tourist attractions, etc. When you visit a big city, you might also want visit its museums. In your exploratory testing, this would be analogous to testing all of the old artifacts that might have been brought forward from past systems into the new solution.

I’m personally a terrible tourist, often trying my wife’s patience. If I’m interested in something, I tend to be totally into it, but if I’m not, I disengage. Think about a tourist like me, and correlate that to the imagined activities of a “disengaged” user. This is someone who is probably not that interested in learning the functionality of the software, may not pay attention to data entry, or may even leave default values in place instead of being diligent in use. Mimic that behavior.

As you can imagine, there are many different ways to organize your exploratory testing “tours” and as many different tours as a person can imagine. Remember, the goal is to validate as much of the application’s functionality as possible, often in constrained timeframes, and to leverage the tester’s creativity, experience, and intuition in the process.

There is some great information about exploratory testing out there. As a person who’s familiar with Microsoft Test Manager, I find there is useful information in its documentation that is very relevant and freely available. Check it out at msdn: Exploratory Software Testing (https://bit.ly/UHZ2pX ).