My Box is Round

Metaphors are cars – I think. There are many metaphors in software, a lot of them are useful, but most break down eventually. In hwtsam (chapter 3 to be exact), I compare creating software with cooking. I’ve mentioned on numerous occasions that testing is like detective work (technically, that’s a simile, but go with me).

In testing, the concepts of black-box and white-box testing have been around for years, and are commonly used in introductory test courses to explain different ways to approach testing. Many explanations go on to say that a “grey box” approach is preferred as it combines both approaches into a more holistic approach.

This weekend, Phil Kirkham asked “What Colour Tester are you?” I whimsically answered “Bright Grey”, but Michael Bolton had a more interesting answer. He denied the existence of the box (which I found kind of strange since he teaches a course called Black-Box Software Testing). Anyway, his elaboration was somewhat obvious, but something that rarely comes out in the box explanation, and was good for the community to see.

Upon reading this, I was immediately reminded of a reply I made in another discussion forum just a few days ago in response to the question “How do you keep fresh ideas flowing”. My answer – which I thought was worth repeating follows.

When I'm feature testing, I try to test from as many different angles as I can (I call this 360 degree testing, but that's just me). If I'm stuck, I'll ask myself two questions: What haven't I tested for yet?, and What testing approaches haven't I tried yet? (sometimes these overlap). My initial exploratory tests may have been primarily looking for issues in functionality or negative input, so next, I may brainstorm on how the font feature may fail (fonts could overlap, fonts could be illegible, wrong font could be displayed, etc.). Then, I'll try to make those things happen in as many ways as I can think of.

When I've exhausted that avenue, I usually change directions entirely and look for memory leaks (I wonder if large fonts make word use more memory? If they do, is the memory released when I change back to a smaller font? What other ways could I cause a memory leak with fonts.

Eventually, I may examine the code or code coverage data (since I work at MS, those are available to me). Gaps in code coverage, or code review may give me more ideas on what to test. Probably much earlier than this point, I'll talk to the developers on the team who deal with fonts, as well as any other font experts I can find to see what kinds of tests they run. This ends up turning into a perpetual circle - by the time I've tried everything I can think of, I've learned enough that I can try the things I tried in the first place, because now I have a new perspective and have slightly different approaches to take.

It seems that I've flowed away from the original question, but I suppose I consider all 360 degrees exploratory (others may argue). I'm learning as I execute tests and changing my approach whenever I get stuck and want to get some "fresh ideas flowing".

So, I suppose my box is actually 360 degrees – which if my math is right, sort of round.

Comments

• Anonymous
  January 12, 2009
  The comment has been removed
• Anonymous
  January 14, 2009
  The comment has been removed
• Anonymous
  January 15, 2009
  The comment has been removed
• Anonymous
  January 16, 2009
  I guess to be a smartaleck about this we should think about what Boris Beizer said a few years back (how many I don't remember, I just remember his comment on a post), "There are no boxes, quite being 'boxed in'.  Think beyond the borders."Which caused a heated debate, but made sense in that he was saying don't limit yourself to one or two techniques/methods and levels of testing in the system.  Expand your view from the local trees to the whole forest.
• Anonymous
  January 17, 2009
  The comment has been removed
• Anonymous
  January 18, 2009
  The comment has been removed
• Anonymous
  January 18, 2009
  The comment has been removed
• Anonymous
  January 19, 2009
  Michael - I stand corrected. I assumed (for a variety of reasons) that you also taught the bbst course. In context of the discussion, I could have said you "were involved" - or looked it up, but apparently I was in a hurry to make a point :}.Shrini - I said "round" for two reasons. One is that since boxes are NOT round, I was attempting to say that boxes don't have to be boxes. The second, more direct reason, is that I mentioned 360 degree testing, and 360 degrees is a circle.
• Anonymous
  January 19, 2009
  The comment has been removed
• Anonymous
  January 19, 2009
  The comment has been removed