K-Based Testing
I was reading Keith Braithwaite's website recently in preparation for asking him Five Questions. One paper in particular caught my eye, wherein Keith describes several case studies where formally specifying a part of a system was difficult or impossible, whereas listing out a bunch of examples of how the feature should work was straightforward. I like this idea, which I've dubbed Example-Based Testing. While some developers argue that their unit tests are the specification, I have seen few unit tests which I can translate into a useful model of a system as easily as I can translate even the most mediocre specification. I am intrigued by the idea of specifying a system by a bunch of examples rather than a bunch of "The system shall" statements.
Keith Stobie, who has been intriguing me with his success in using model-based testing to test Windows' protocols, recently told me that his team was doing automated testing. I was a bit bemused at this, because automated testing is rather a Big Thing here at Microsoft. Why would Keith make a Big Deal about it? He replied that while most Microsoft testers execute at least some of their tests automatedly, most of them still generate their tests manually, by hand coding them. The model-based testing tool his team uses both generates and executes their tests automatedly - true automated testing.
I remember Harry Robinson sketching a future where testers spend their time expanding models which generate test cases rather than generating test cases themselves. I wonder how Example-Based Testing would fit into this world. How would Keith have modeled those systems whose workings he could define only by listing examples of valid and invalid results? This enquiring mind wants to know.
As I write this I am sitting in a Seattle park on a rare-this-summer sunny day, watching dogs gambol about and failing at not watching a man wearing only what appears to be a g-string. Watching dogs and people fascinates me. Watching testers and their varied-as-the-people-in-this-park approaches to testing also fascinates me. I have yet to spend time with a tester and not learn something new. Talking with testers and learning too - I cannot think of a better use of my time!
How about you? What new testing technique have you experimented with recently? How did it work? Let me know!
*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.