What Tests Belong in the BVTs?

BVTs or Build Verification Tests are standard Microsoft parlance for the tests we run every day to ensure that we didn't break anything important with our checkins the day before.  I've previously written about the importance of keeping them clean.  Within the range of tests that consistently pass, which ones should be in the BVT?  BVT test failures should be something you're willing to act on immediately.  In other words, the failures must be important.  Based on that, here are some criteria:

• Test major scenarios not minor ones.  If major features are failing, they will be fixed right away.  If a minor feature is failing, it should be noted, but may have to wait until later to be fixed.
• Test majority use cases, not corner cases.  Tests for the interaction of 3 parts shouldn't be in the BVTs.  Tests outside most user scenarios shouldn't be in the BVTs.  While every book on testing says to test the boundary conditions, the BVTs may not be the place to do that.  Instead, pick the most likely to be used values and scenarios.
• Run "positive" not "negative" tests.  By that I mean, don't send out-of-bounds conditions or invalid values.  These are valid tests and should definitely be run, but not in the BVTs.  An API faulting when sent a null pointer should be fixed, but the fix can wait until next week.

BVTs should be a carefully guarded set of tests.  They need to run quickly, consistently, and their results should matter.  If these rules are followed, the BVTs will be effective because failures will be respected.  Restricting the BVTs to the most important scenarios will ensure that the results are given the appropriate respect.

Comments

• Anonymous
  March 17, 2008
  文章描述了BVT测试的一些原则。