Code Coverage (CC)

There are people out there I know who believe that hitting a Code Coverage goal of 90% is a good reason to stop testing. Now given that I have to admit that hitting a 90% code coverage rate is good, it however should not be used as the sole reason to stop testing.

Using code coverage as a sole measurement to know when to stop testing is wrong because using code coverage numbers alone doesn't indicate how well the tests used to obtain the coverage results executed the code and whether or not the code was executed in the proper scenarios required for hitting the release criteria. It could be that the 10% missed is the code that 80% of your customers need to use and it wasn't executed at all. Or it could be that the 90% hit was not executed in the proper order of execution for the scenarios that 80% of your customers will execute it.

So, it is very important in the planning stages that PM, Dev and Test properly communicate and set code coverage goals for the milestone that the team can drive towards and measure progress by. It is equally important to define and prioritize the key scenarios used by your customers and to insure that the test cases and scenarios that execute the code cover the scenarios used by at least 80% of your customers otherwise, you will not know if the proper code was executed in the correct manner… you will just be looking at code coverage results.