The value of Manual testing (or…The value of Automated testing)

I’m skipping #8 for a minute so I can get to #9.

I hope we all agree that it’s silly to take a perfectly find manual test and automate it. It’s also silly to take something that’s highly repetitive and easily automatable and NOT automate it. So, as is the theme for this series, let’s discuss what happens when your org doesn’t get it.

On one extreme, your org may want to manually test everything. Depending on the situation, it may be a reasonable choice. If it’s not, do something about it.

I’m reminded of one of my first test assignments. I was kind of a programmer. I could write simple C programs, but I kicked butt writing automation with MS-Test (the precursor to Visual Test). I wrote a massive number of questionably designed, yet very_cool_to_watch automation at my previous job and “thought” I was hired to do more of the same.

So, I walked into my boss’ office, he handed me a list of test cases and told me to get to work. I glanced through the list, asked a few clarifying questions, then asked when he wanted I should finish automating them. He replied “oh no, we don’t have time to automate – you just need to run these every day”.

I was young and a little scared to rock the boat, so I did what he asked. That was Monday. By Wednesday I was bored, and by Friday I had automated most of the test cases. Not only did I not miss any regressions during the remainder of the product cycle, I actually had time to do testing!. Eventually, I clued my manager in on what I did (he had grown to trust me by this point), and he liked it. In fact, he asked me to automate some fairly complex scenarios (which I did), and we eventually ended up with a good mix of manual and automated tests.

I guess the moral of the story was that I just did what I thought was right and got away with it. A safer solution would be to ask first, but a small pilot project - even if it’s on your own time could probably help here.

Sometimes, the opposite happens. Teams are so focused on automating everything that they don’t actually do much testing. In this case, just do the opposite of my example above. Stop writing automation for an hour or two every day, or for 6 hours on Friday, or whatever, and use the product. If you find cool bugs (and if you’re any good at testing, you will), note that they were found by some technique other than automation. Eventually, people will notice that you’re finding lot’s of cool bugs (don’t worry, you’ll keep up on your automation work too), and they’ll ask what your “secret sauce” is. Tell them the truth. Soon, they’ll be enough sauce flowing to actually whip your product into shape.

Cool stuff.

Comments

• Anonymous
  September 04, 2008
  PingBack from http://informationsfunnywallpaper.cn/?p=3801
• Anonymous
  September 05, 2008
  Another approach is to ask your manager for assistance coming up with realistic numbers for CALCULATING whether automation is really justified or not.  I've used this spreadsheet as a simplistic way to show at what point it makes sense to automate:http://www.primarygoals.com/Technology/automation.xlsThe biggest shortfall has been when one thinks "we'll only run this test maybe 10 times," but because it keeps failing, it actually takes 20 more passes before it really works.  This creates the pre-automation view that it would not make sense to automate, but the post-testing realization that it would have made sense to do so after all.
• Anonymous
  September 05, 2008
  The comment has been removed
• Anonymous
  September 06, 2008
  Thanks for the comments.I absolutely agree that the key is to have a good automation strategy that makes sense (although to me, Bach's post falls into the ("please tell me something non-obvious" category). The real point of this post is how to fix your automation strategy if it's broken (as many seem to be).So, don't tell me what a good automation strategy is, tell me how to fix it. Shrini - I hear you complain about bad automation strategies often - other than flat out telling people they're wrong, what do you do to fix it?
• Anonymous
  September 06, 2008
  The comment has been removed
• Anonymous
  September 06, 2008
  No organizational change is easy - especially if you're anywhere but at the top of the organization (and even that's not a guarantee).But I'm not sure that refocusing on testing will make a difference. You need to show proof, then you need to show it again. Merely explaining why one approach is better than another is never going to change anyone's mind.
• Anonymous
  September 10, 2008
  Alanpa on The value of manual testing (or...The value of automated testing) Charles Sterling on New...