Share via

Bug Triage

  • Article

Eric Sink has a good article on the process of triaging bugs. In it he pokes a little bit of fun at the people who don't understand why all good software ships with known bugs and then discusses a process for determining which ones to fix.

Determining which bugs to fix and how to prioritize them is often a challenge, particularly for bugs in the middle of the severity/frequency ranges. There's no objective algorithm, so as Eric's article suggests it often boils down to a couple people discussing the bug and making the call.

However, there's another subtlety: certain classes of bugs often have strong advocates, including external teams tracking metrics and driving to get those bugs fixed. This puts extra weight behind those classes of bugs, which can be both a good thing and a bad thing. The good part is that in general these are severe bugs which the team would to fix anyway, and the metrics and pressure to get them fixed helps ensure that the team is shipping a product with known high quality. However, the downside is that this removes some amount of control (both consciously and subconsciously) from the product team, which can sometimes result in the wrong bugs getting fixed.

As an example, let's imagine that there are two bugs open, and the team only has time to fix one of them before a beta release ships:

1) Crash

2) UI element doesn't behave quite right

Crashes are a class of bugs which always rank extremely high on the priority scale, are tracked by metrics, and have strong advocates to get them fixed. In the absence of other information someone triaging these bugs will almost certainly rank #1 as a higher priority bug than #2.

But wait a minute: there is more information. The crash only happens in an obscure feature, and has only been observed while stressing the product using automated tests. There is no known way to manually reproduce the crash (the people who work on the feature can't get it to crash on demand), the fix isn't straightforward, and it doesn't appear to be a security risk. The fix might even be risky, or mask a more serious bug. Furthermore, suppose the UI element is in a high-profile feature, and that feature not behaving correctly will grate on the beta tester every hour that they use the product.

Now which bug do you fix? Clearly you fix #2.

This becomes obvious once you have enough information to make an informed decision, which is the key. Bugs cannot be triaged blindly by objective criteria; they must be judged by people that are not only familiar with the product and feature, but also have sufficient information about the bug to make an informed decision.

Comments