Freigeben über

Continually Improving Tests

  • Artikel

I had a conversation with someone this week about how to improve the efficiency of our automated test runs.  One thing I suggested was going through our test suites and getting rid of obsolete, redundant, and troublesome tests.  I expected to get pushback around us not having resources to do the work but instead the pushback was around getting rid of any tests at all.  Apparently, some folks think that if you reduce the number of the tests you run, you reduce the effectiveness of your testing.  I respectfully disagree.  At some point in time, the tests will grow to an unmanageable number.  Before that, you'll find your testers doing nothing but maintaining your huge body of existing tests when they should be testing the product.

In my ideal world, testers would always be cognizant of the cost of a test as well as its benefit.  They would also be continually improving their tests - refactoring them, consolidating them when appropriate, removing redundancies.  They would always know the content of their test suites and where the trouble spots are.  Tests for an area would be factored into the appropriate layers (more on this in a future post) based on the layer's purpose.  The tests suites would be improved based on feedback like missed product issues, peer review, and increasing knowledge of the product.  Fixes wouldn't be band-aids over symptoms but rather would be done with a consideration of the architecture of the test suite and framework.  The test automation would be continually improving rather than gradually giving way to entropy.

I realize this is an ideal world but I'd like to think we can move in that direction.  A good place to start is where you currently are.  When you create new tests keep these ideas in mind.  When a test breaks, fix it the right way rather than the applying a band-aid over the symptoms.  When new feature work happens, create new tests with consideration to the existing tests for that area - maybe even rework those existing tests to handle the new feature behavior.  Remove tests that no longer apply.  Gradually we can move to a place where we have enough confidence in our tests that we (and others) won't be afraid when their number goes down.

Comments