Jaa

Testing is NOT responsible for quality!

  • Artikkeli

The value of testing is important and critical to the success of many projects. However, testing is NOT solely responsible for the quality of a project! 

When we discuss the role of testing and quality in our internal training we emphasize the primary objectives of the tester as providing information and assessing (preferably via quantifiable measurements) quality (and specifically focusing on the quality traits or attributes that are most crtical to the success of the project).

But, there are still some managers who would like to believe that the testing group within an organization is responsible for quality. And there are even some testers who will tell you they are responsible for quality. I won't begin to speculate how this folktale came to be, or why it is perpetuated. But, it is a folktale (any belief or story passed on traditionally, esp. one considered to be false or based on superstition). Folktales usually persist in societies for a variety of reasons. There is no doubt that quality is important. But, quality is hard to define; quality is hard to measure, and it is hard to translate quality into value for the customer. There is no doubt that as a tester I can influence the quality; however, responsibility implies the ability to also control. How much control does a typical testing organization in a commercial software company actually have upon the quality of a project?

There are 3 simple tests to refute the conjecture that testing is responsible for quality.

  1. Can we test in quality?
  2. Can testing find all the defects?
  3. Will all the defects be fixed? (Here a defect implies deviation from explicit or implicit requirements or expectations, or unexpected behavior.)

The answer to these three questions is no! Unless you are an absolute novice to software engineering you will probably agree we cannot test in quality. Testing cannot find all the defects in any complex software project (and there is ample evidence of this). And anyone who has any experience on a project of significant size will admit that not all defects are fixed (including functional defects). So, as a tester I know I can't test in quality, I will never find all the defects, and I know that some of the defects I find will not get fixed (because I can't use electric cattle prods on developers and I am not going to go into the code and fix the bug myself), so how in the world can I be responsible for the quality of a product?

As a tester I am responsible for providing information regarding the percieved quality and/or risk of a project as assessed by a series of tests, observations, and experiments, and that information is provided to managment (who hopefully use it) to determine whether a commercial software product is released to the public.

So, who is ultimately responsible for the quality of a product? Often times folks will rally around the "everyone is responsible for quality" mantra. But, W. Mark Manduke wrote an nice article in STQE magazine a few years ago stating "When quality is declared to be everyone's responsibilty, no one is truly designated to be resonsible for it, and quality issues fade into the chaos of the crisis du jour." He concluded "...when management truly commits to a quality culture, everyone will, indeed, be responsible for quality."

Thus, ultimately the management team (the decision makers) is responsible and accountable for quality.

Comments

  • Anonymous
    April 10, 2007
    The comment has been removed

  • Anonymous
    April 11, 2007
    The comment has been removed

  • Anonymous
    April 22, 2007
    Hi William, This is a good post. The comments that followed are also interesting. In a Banking and Finance vertical of an organization, not to name, the weekly target for a tester is not in terms of number of test cases or otherwise, rather in terms of number of bugs that he finds in a week. I always feel sad about the testing guys there. I guess the management there should read this post of yours. Regards, Rahul Verma.

  • Anonymous
    April 24, 2007
    Hi Rahul, Thanks for your comment. I previously posted about the misuse of bug count as 'targets' or as evaluation criteria for testers. The title of the post is Bug counts as key performance indicators (KPI) for testers (http://blogs.msdn.com/imtesty/archive/2006/06/26/647628.aspx). (for some reason the insert/edit link function on the community server seems to be broken today.) I think you will enjoy that post as well.

  • Anonymous
    April 24, 2007
    The comment has been removed

  • Anonymous
    April 25, 2007
    The comment has been removed

  • Anonymous
    July 30, 2007
    Good article on Testing vs. Quality. My comment is, that your are right that WE - the people with the competences within testing, are the ones who should "teach" the managers what testings is about. If they do not study this themselves (and they probably don't) - we really have to help them. We, as in companies, managers, testers, test managers etc. - do not improve in this area, these tasks of developing projects, if we do not try to teach "newcommers" - managers and actually also people from business, what testing is about, how it should be done - and not least, the outcome to expect from it. We have that responsibility. cheers, Anders

  • Anonymous
    August 01, 2007
    The comment has been removed

  • Anonymous
    June 10, 2008
    上一篇談 Tester 的職責是什麼, 這一篇談另一個觀念, Tester 不應該被叫做 QA (quality assurance), 因為基本上, Tester 沒辦法真的確保 "Quality".