다음을 통해 공유

the manual v. automated testing debate

  • 아티클

There's an angle to this debate that I missed during the prevention v. cure series I did last month. It surfaced in a lunchtime conversation I had today with two test managers in our e-home division (these are the guys that test the Media Center PC and other such delights). Michael Friend, a Group Test Manager who's been around the empire far longer than I have, gave an interesting perspective: "you know automation has gained too much of a foothold when your testers feel more vested in their test code than in the product they're shipping."

Ouch, that was aimed at me.

I'm often very proud of my own test code and some of the tools I've built have, to me, been far more compelling than the code they've tested. I've walked this line Michael talks about and apparently fallen on the wrong side. Michael's point was that manual testers, by definition, are closer to the product they are shipping than automation experts. In manual testing we're hands-on and directly involved; in automated testing we build automation that touches our product ... we are a step removed from it. I wonder if there is something psychological at work here that gives manual testers an advantage on passion, insight and product familiarity that makes them more qualified to find bugs. Certainly, I think that connecting closely with your product will make you a better tester and Michael himself said he prefers testers who find the right balance between automation and hands-on testing. The more hands-on testing they perform, the better their automation.

Brent Jensen, who doesn't let his youth stand in the way of being old school, said it best: "when testers treat their automation like they gave birth to it, they've crossed the line."

Parents are often the only ones that think their baby is beautiful. How often have you been bold enough to say, "man that is one uuugly baby!"

Comments

  • Anonymous
    September 05, 2008
    PingBack from http://www.easycoded.com/the-manual-v-automated-testing-debate/

  • Anonymous
    September 05, 2008
    Debating manual v. automated testing is like debating eating v. drinking.  You can't survive without either one. Automated testing frees the tester to do more manual testing.  It also helps the tester stay passionate about the product she is working on.

  • Anonymous
    September 06, 2008
    The comment has been removed

  • Anonymous
    September 07, 2008
    Putting the 'v' between the two does imply a contest. But it isn't to be viewed as an 'or' but the tradeoff and pros/cons and getting the mix right. Good comment ... thanks.

  • Anonymous
    September 07, 2008
    I want to respond to the comment on comparing manual testing and automation testing to eating vs drinking while pointing out a subtlety or a distinction based on what is being automation. If ur automating a set of API's or driving an object's methods to get test coverage I can see a lot of code(and coding) being involved. In such cases, i can see how a tester could get carried away with the testing code more so than the product itself. On the other hand, if you are automating user clicks or end user actions then really I don't see how you could automate without actually having manually exercised the application/user actions or at least explored the application and made decisions on what you want to automate. Such approach to automation may not exercise all the methods, but only the ones exposed by the end users actions. It ofcourse does have a down side. From the POV of automating end user actions, isn't manual testing a sort of a pre-cursor to automation (of user actions). If you got a brand new app to automate, what would you start doing? Ofcourse, I'd eat and drink for sure before embarking on any work.

  • Anonymous
    September 08, 2008
    The comment has been removed

  • Anonymous
    September 09, 2008
    The comment has been removed

  • Anonymous
    September 09, 2008
    The comment has been removed

  • Anonymous
    September 12, 2008
    I'd like to hear the things that make you happy in a review calibration! I actually liked your comment better than you liked my post.

  • Anonymous
    September 15, 2008
    The comment has been removed