次の方法で共有

Dev owns testing, Dev owns quality, really?

  • [アーティクル]

You might hear a lot about dev owns quality recently. James whittaker, the former engineering director of Google, spends quite a lot pages in his new book <<how Google Test software>>  to articulate why dev should own testing, and why should dev owns quality, like this:

“Who better to do all that testing than the people doing the actual coding? Who better to find the bug than the person who wrote it? Who is more incentivized to avoid writing the bug in the first place? The reason Google can get by with so few dedicated testers is because developers own quality.”

However, not surprisingly, there are arguments on this. Some, particularly fulltime tester/QA believe dev doesn’t have mindset or required skill set to do good testing. On other hand, you can’t ask one to test their own codes. Human is very bad on verifying her own work. Asking dev to test her own codes is like asking a wolf to guard sheep. It just won’t work.

…

…

Really?

…

Before we jump into any conclusion, let’s look at how things got evolved with a coding example. Assume dev was asked to provide an API to calculate tax based on the income, factor and a value stored in database. (disclaimer: the code example is to make a point with obvious bugs, no offense to dev :))

In old days, dev doesn’t test her code but simply hands over to tester for quality assurance. The code very likely looks like this:

class Tax {

     ......

     public double CalcTax(int income, int factor)

      {

            DBConnection c = ConnectToDB();

             Int t = c.GetValue();

            Return income/(factor*t);

    }

  }

Why? Because dev’s role is to writing products codes to build the features, not assure its quality. Quality is tester’s responsibility, so I don’t need to care and I don’t want to care. My performance is measured against how many lines of codes I write (or how many new features I build).

 

Now, dev needs to do the testing… the code probably looks like this:

class Tax {

      ......

     public double CalcTax(int income, int factor)

      {

            if factor == 0 then return 0;

             DBConnection c = ConnectToDB();

             Int t = c.GetValue();

             if (t==0) t=0; // this is bug.

            Return income/(factor*t);

    }

}

 

assume dev made a mistake, it actually want to write if (t==0) t=0.1; the bug will stay undetected, due to its dependency on values in database. if there is no zero in the test db, the bug probably slip to customers that has zero in their db.

 

Then, dev owns quality... the codes probably looks like this:

create fake database mock.

TestCase1.CalcTax(100, 1);

TestCase2.CalcTax(100,0);

TestCase3.CalcTax(100,1) with mock db to let GetValue() return 0;

.....

 class Tax {

        .....

     Database db;

     public Tax(Database d) { db = d;}

     public double CalcTax(int income, int factor)

      {

           if factor == 0 then return 0;

            DBConnection c = db.ConnectToDB();

             Int t = c.GetValue();

            if (t== 0 ) t=0.1;

            Return income/(factor*t);

    }

}

 

so, what we learned?

  1. Dev owns quality means better codes.
  2. Dev owns quality means find bug easily.
  3. Dev owns quality means find bug early, much earlier. We know the later you find a bug, the more expensive to fix.
  4. Dev owns quality means reduce the hand off, more efficient and more fast.
  5. Dev owns quality means test ability. When dev owns quality, dev has to think about how to test it when write code. They can’t be tested manually. Test automation is the only way to go, otherwise, he is not going to have time to write new codes any more but doing manual testing all the
    time :)

 

Here,you might think this is TDD. Dev owns quality means use TDD. Well, not exactly.  My experience with TDD is more in the ideally world. I don’t see any team yet use pure TDD. Most teams use what we called delay TDD. You still write code first, and then write test, then do the refactoring, in the end submit both code and test at the same time. This is more nature way to develop software and achieve the same goal of TDD, meanwhile it’s much easy to adopt vs pure TDD which is completely opposite of what you came along. furthermore, dev owns quality can be extended to owning functional testing, i.e not only making sure unit works by unit testing, but also make sure component works by functional testing.

 

What’s the cost of dev owning quality? “Slowing down?” Yes or no. the problem is that we only see the cost of doing unit tests, but not see the cost of not doing it. since dev need to writing a lot of testing which takes time. Of course, she will spend less time writing product codes, which eventually slow down building new features. However, if you see the whole application life cycle, developing feature is just one phase of it. There are maintenance, livesite, and application getting more and more complex. Some industry researches show that it actually shorten application life cycle, because without early testing, a lot of bugs are found at the late stage and even in the customer’s environment, this start to eat dev’s time and eventually slow down dev. in contrast, dev owns quality makes software component robust at the first place. It’s like building a highway v.s. bumpy road, initially car goes faster on bumpy road since it takes time to build highway. But once highway is ready, the car goes much faster than running on bumpy road.  This actually leads to another question that is there any situation that unit testing doesnot doing any good? yes, there is. again, take the high way as an example, if you go that road only once or twice, you probably don't want to build a highway. however, if you have to go on the road again and again for the next 5, 10 years, i think you better start to build highway.

 

By far, I hope you are with me that dev owns quality is a good thing. But what if dev agree but still don’t do it? Many students told me that “we all know /agree this in theory, but dev just don’t do it for whatever reasons, how can I change it?” Here I’m going to share with you a few tips you can try.

First off, there is no silver bullet for it. Dev born to be writing product code, they don’t like testing  in nature. It is a hard and long effort to make the change. So you got be prepared (hopefully you are not discouraged). On other hand, challenge goes with opportunity. Based on my  experience, the change from the state that dev refuse to testing  to the state that dev love testing, are not one step task, it usually take a few phases.

1. awareness. Your first task to convince dev that dev owning quality is good thing or industry trend. If they agree, you are half way to success. if they don't agree, any effort you try to push is useless. so, you got to get them on board first. no exception.

2. make unit test as part of code complete criteria. Once dev agree it’s good thing to unit testing their own codes, you can start to make rules on it. For example, we must have unit tests for every API before we declare code complete. Here is tricky part, at the end of the first iteration, very likely you still don’t see any unit test.  That’s ok, make an exception, but the key thing is let the team knows we are make an EXCEPTION this time. I bet at the beginning of second iteration, dev will start to consider unit test into their overall work. Hopefully at the end of 2nd interation, you will see a little unit test. Again, it's not good progress but it’s good sign. Just be patient. Make another EXCEPTION and let the team knows. So on and so forth, after a few iterations, you will suprisely see there are a lot unit test in place.

You also can try peer pressure or indirect push tips to make the unit test coverage transparent, so the team knows who write unit test, who doesn’t.

Reward those devs or teams having better unit tests. measure dev's performance based quality of codes, not number of lines of codes. build a culture that team take pride of their unit testing results.

Working with dev to improve unit test framework and make unit test as easy as possible. Image you are devs, you take 1 minute to write one line of product code, and takes 2minutes to write its unit tests, now takes 30 minutes to run unit tests. You won’t do it either, right? So the most efficient way to push dev to do unit test is to make it as easy as possible.

3. make it default. After the team is armed with tools/framework for unit testing, you can make it default. For example, block the checkin if no unit test. Showing unit test cc before checkin, etc…

 

If dev owns quality, what a tester will do?  Will fulltime tester be phased out?  i will try to answer these questions in my next blog, also talk about tester's core competency, so, stay tuned....

Comments

  • Anonymous
    February 02, 2013
    你好，这是原创吗？我在等你的next blog ，去哪了？
  • Anonymous
    February 02, 2013
    当然是原创。 还没时间写下面的
  • Anonymous
    February 03, 2013
    嗯嗯好的，保持更新哦，写得很不错！