Automation as inoculations

One of the books I have been reading is Visual Studio Team Foundation Server 2012 : adopting agile software practices : from backlog to continuous feedback [sic] which is quite a mouthful of title, if nothing else.

(The tester in me is going to go on a quick tangent here. According to my second grade teacher, Mrs. Spurlin, major words in book titles must be capitalized. Looking at the text I frankly copied/pasted above, I see the title of the book is incorrect. But when I look at the actual book cover, the text is capitalized properly. Here's a snippet of the UI from our library site.

So now the tester in me wonders how this text got garbled when the book title was added to the library. Was it a scanning error? Did someone physically type the book name into the database? Did some errant spell checker decide to change some of the letters to lower case? The tester in me is bothered by little errors like this…)

Anyway, one of the points of view the authors take is that you can think of automated testing as inoculating your product code from certain defects. A simple example would be the napkin math feature in OneNote. We have an automated test which adds 6+4 and verifies answer is 10. Now that we have this automated, if we ever change anything that affects 6+4 being equal to 10, we will know. In some sense, we are now immune to that bug since we can detect it and fix it when we make the breaking change. But we are not (if that was our only test) immune to breaking subtraction, since we presumably have not tested 6-4=2. So one of the takeaways I get from this viewpoint is that you need to know what you have and do not have automated. This can help tell you where risk exists when you start changing things.

This is a slightly different point of view of "automation as insurance." The primary challenge when treating automation as insurance is knowing when you have enough. But if you shift your viewpoint to inoculations, you can maybe still not know when you have enough, but you can tell where you need more. I'll stick with the insurance analogy and say that if I know I have 100% coverage on my bicycle (analogous to napkin math) but only 50% coverage on my car (any other feature in OneNote, let's say Format Painter) then I probably need to make my next investment in car insurance (format painter automation).

I think this was only one paragraph in this entire book, but it gives me a lot to think about. For instance, how to know you have 100% coverage? Or 99%? Not an easy question to answer. Like I said, a lot to think about.

Questions, comments, concerns and criticisms always welcome,

John