tour of the month: the landmark tour

Every location that covets tourists must have some good reasons for them to come. For Las Vegas it’s the casinos and the strip, for Amsterdam it’s the coffee shops and red light district, for Egypt it’s the pyramids. Take these landmarks away and the place is no longer an attraction.

Software is much the same – there has to be some reason for users to buy it. Some landmarks that make it what it is. If you identify the landmark features that draw users, that’s where you need to concentrate a lot of testing. You can tell where the tourist landmarks are by noticing crowds and watching where they spend their money. For exploratory testers finding the landmarks also means following the money, literally. And money usually leads directly to the sales force.

Sales folk spend a great deal of time giving demos of applications. One might imagine that since they get paid based on fulfilling their sales quota, they would be very good at it and would include any interesting nuances of usage that make the product look its very best. They also excel at shortcuts to smooth out demos and often come up with scenarios that sell the product but weren’t part of any specific requirements or user story. The long and short of it is that sales people are a fantastic source of information for the landmark tour.

Testers performing the landmark tour should sit in on sales demos, watch sales videos, and accompany salespeople on trips to talk to customers. Each demo or demo script identifies the landmark features and can even give you insight into their relative importance by how often those features appear.

To execute the tour, you could simply run through the demos yourself and look for problems. As the product code is modified for bug fixes and new features, it may be that the demo breaks and you’ve not only found a great bug, but you’ve saved your sales force from some pretty serious embarrassment (perhaps even salvaging a sale). I have found enough bugs this way to privately wonder if there is a case to be made for testers sharing in sales commissions!

But we’ve found a more powerful way is to inject variation into the demo scenarios by “landmark hopping.” Ever use a compass in the woods? That’s the effect we are going for here. My army-trained older brother taught me to locate landmarks (rocks, trees, hillsides) in a compass scope and then to make a path from landmark to landmark in the general direction you want to go. It kept me from getting lost in the woods of Kentucky where I grew up and it creates excellent variation in demo scripts and end-to-end sceanrios. Pick the landmarks in advance, scramble them up and tour from one to the next. Rinse and repeat trying to use the landmark feature in a different way every time.

A powerful variation of this tour is the skeptical customer tour in which you execute the landmark tour but pretend there is a customer constantly stopping the demo and asking ‘what if’. ‘What if I wanted to do this?’ they may ask, or, ‘how would I do that?’ requiring you to go off script and include a new feature into the demo. This happens a lot in customer demos, especially the serious ones where a purchase is imminent and the customer is kicking the tires one last time. It’s a powerful way to create test cases that will matter to end users.

Landmarks can be "back of the box" features, features used as part of sales demos, features popular to users (in the event you collect such statistics), features prominently displayed in the UI and so forth. Clearly any bugs you find on this tour or any of its variations are very important ones as they are likely to be seen by real customers.

Enjoy the tour and don't get lost.

Comments

• Anonymous
  May 11, 2009
  James - come back to the blogosphere. We miss you