Freigeben über

How we prioritize bugs

  • Artikel

As WPF gets closer and closer to shipping, one of the things I spend a lot of my time doing is bug triage -- prioritizing bugs and deciding which to fix. We'd like to fix all bugs, of course, but that's just not reality. There's only so many hours in the day, you can only throw so many people at a given piece of code, and fixing bugs often creates new bugs. So it's important for us to be selective. And we’d love your feedback in doing this.

Every team in WPF is a little different, and so prioritizes bugs a little differently. But my element services team (parser, property engine, styling/templating, input/commanding, interop) looks at three main factors. First thing is who reported it, and how important the bug is to them -- something that was found by you is going to get higher priority than the same issue if we found it internally writing an artificial test case, because we know at least one real customer will actually hit the bug. We’d love as much information as you can provide – not just what went wrong, but what you’re trying to accomplish, why you’re trying to accomplish it, and even in some cases the business goals of your product – the better we understand where you’re coming from, the better we can understand the real impact the bug is having on you and other people.

The second thing we look at is what the impact will be on future versions of WPF, the "now or never" factor. This comes up a lot with making sure the xaml parser doesn’t accept invalid xaml – because that would effectively change the xaml spec. And that’s a problem because there’s more than one xaml parser in the world, for example Expression has its own parser (because design-time requirements are different from those of a runtime parser).

It’s easy to overdo the now or never factor – pretty much anything, once shipped, is difficult to change, and some of those imperfections will have much more impact than others. Generally speaking, API cleanliness issues are quite low on our priority list at this point – eg, calling it "IsFocusable" instead of "Focusable". My goal is to ship a good API, not a perfect one, because if we tried to ship a perfect API, we’d spend so much time on the API that we wouldn’t build useful, performant functionality.

The third factor is the difficulty of someone outside the team understanding what's going on and creating a workaround. Some things are pretty easy to understand, others are fiendishly difficult. Generally speaking, the lower the bug is in the system, the harder it is to understand – sometimes to the point that we can’t even guess who will be impacted. For these reasons, we generally take race conditions and invalid results from the property engine and input system as very important bugs. Predicting how other people will understand something is obviously a tricky business, I do my best to calibrate this based on your newsgroup questions so keep 'em coming!

Final thing to throw in there is I believe in being honest about whether we’re going to fix something. As I said, I would love to fix everything – but I know we aren’t, so if I don’t think we’ll get to it, I’ll resolve that bug "won’t fix". I’d rather you find out now than later, because that will give you a chance to let me know if I made the wrong decision.

For specific bugs, ladybug is the preferred approach. If you have more general concerns about categories of bugs or priorities, this blog or my e-mail is a good forum. It’s all about making the right trade-offs for you, so we appreciate the feedback!

Comments

  • Anonymous
    March 03, 2006
    I feel it's better to change title from "How we prioritize bugs" to "How we postpone bugs".

    This is more fair for users.

    Setting priorities is about to order things that will be done, but in MSFT - it's about things that will NOT be done (in this release and probably never).
  • Anonymous
    March 17, 2006
    It's true that some things will not ever be done, as long as the number of things to do exceeds the resources available to do it. Some things will never get done before release, because the benefit of releasing the code

    For example, Windows XP has many widely-known holes in it. Probably some of them would have been fixed if Microsoft delayed release for another year. Would it have been better to do that and leave people with Windows ME? I don't think so.

    XP SP2 isn't perfect, either. Is it better than the original in terms of security? Certainly. Should people have had to wait until now to get it to ensure that even more problems were fixed, or maybe to add more new features along the way? Of course not. There was a need for a fix now.

    Is Vista going to have problems? Sure. Some of them will probably be known beforehand. Do the benefits of releasing outweigh the problems still to come? That's Microsoft's decision. It's not an easy one, but there's a simple solution - if you don't trust it, you can always wait a year or so and see then. Remember they've already gone for three years longer than they originally intended.

    And if you don't like their method of prioritizing you can seek to change its criteria, but you're not ever going to get away from the basic fact that some bugs simply aren't worth the time to fix them . . . and nor are some features worth time implementing. Microsoft has to pick and choose. Personally, I don't envy themm, but hey, that's why they make the big bucks. :-)
  • Anonymous
    March 19, 2006
    "Setting priorities is about to order things that will be done, but in MSFT - it's about things that will NOT be done (in this release and probably never). "

    That's the way it is at any company that.... well, does anything.   I've never worked on any product in any company (I've never worked at MSFT) that management really wanted "perfect" - they want it good and cheap, preferably fast as well.  

    If you have a software project of any useful size, how can you ever fix every issue?  You can't - prioritization is about determining where the cost of the fix (in time and money) is worth the benefit or is justified by the risk.


    Heck, this isn't just software - it's your house, car, TV, even your doctor does this.

    Now, if your issue is about the actual prioritzation methods, or about priorities of specific issues that you disagree with, then say that... Many will probably agree with you.
  • Anonymous
    March 20, 2006
    One of the great joys of my role at Microsoft is that I get to work with some really cool partners who...