Prioritization

아티클
04/28/2015

By Mitch Lacey. Owner, Mitch Lacey & Associates, Inc, a consulting firm specializing in agile and scrum adoptions and improvements.

January 2012

In this article, Mitch Lacey discusses three methods that have proven very beneficial for many Agile teams: the Kano Model of Customer Satisfaction, a series of Innovation Games by Luke Hohmann, and Karl Weigers’ Relative Weighting model. Any of these can help you move from rough prioritization of your backlog to a precise ordering that satisfactorily weighs risk, importance, and customer satisfaction.

Applies To

Application Lifecycle Management; Team Foundation Server

In this article:

Introduction
Kano Model of Customer Satisfaction
Innovation Games
- Prune the Product Tree
- Buy a Feature
Relative Weighting – Karl Weigers
Conclusion

To keep your agile team functioning effectively, you must order the items in your product backlog by priority and then update those priorities as the project progresses. All product backlogs must be prioritized based on business value and risk. By recognizing this priority order, your team can better focus on the features that are most likely to factor into the success of your product. A well-ordered and prioritized backlog pays off not only in team and customer satisfaction but also in your business’ bottom line. For information about backlogs, see Build and manage the product backlog and Create your backlog.

When ordering and prioritizing your product backlog, you must consider many factors as well as be able to justify your decisions. When you’re starting with a raw product backlog, the process can seem almost overwhelming. Fortunately, you don’t have to perfectly order every item in your backlog right away. In Estimating, I describe a technique called “The Big Wall,” which is an effective way to quickly estimate a raw product backlog and work with stakeholders to arrive at a rough prioritization.

This rough prioritization is a good starting point and might be good enough for your circumstances. In some cases, however, you might want to refine these priorities or arrive at a prioritized backlog in a more scientific way. You can accomplish this by using many techniques. In the following article, I discuss three methods that have proven very beneficial for many Agile teams: the Kano Model of Customer Satisfaction, a series of Innovation Games by Luke Hohmann, and Karl Weigers’ Relative Weighting model. Any of these can help you move from rough prioritization of your backlog to a precise ordering that satisfactorily weighs risk, importance, and customer satisfaction.

Kano Model of Customer Satisfaction

The Kano Model of Customer Satisfaction was developed in the 1980s by Professor Noriaki Kano of Tokyo Rika University. His model provides a simple ranking scheme that distinguishes between essential and differentiating attributes. A questionnaire-based approach, the model is a powerful way of visualizing product characteristics and stimulating debate within the design team.

Example product functionality graph

In Kano, we ask a series of questions in two different forms: functional and dysfunctional. For example, let’s say we are asking customers about a GPS navigation system for cars. We first ask the functional form of the question:

How would you feel if this car had a GPS navigation system?

We limit the responses to the following answers:

I’d like it that way
I’d expect it to be that way
I am neutral
I could live with it that way
I’d dislike it that way

For this example, let’s say that our fictional customers responded with “I like it that way.”

Next we ask the dysfunctional form of the question:

How would you feel if this car did not have a GPS navigation system?

Our fictional customers can choose from any of the answers listed. However, the answer can often be, and usually is, different. For this example, let’s say that our fictional customers responded “I expected it to be that way” to the dysfunctional form of the question.

When we do this for an actual project, we can ask this list of opposing questions to multiple customer groups, that is, different sets of individuals who represent different functions of the customer organization. You might have a marketing group, an accounting group, a manufacturing group, and so on. For the purposes of understanding the model, however, let’s imagine that we ask just one question of just one customer group. After we compile our answer pair (the answers to the functional and dysfunctional form), we map it onto a grid, as Table 1 shows.

Example Kano Mapping

Table 1 – Kano Mapping

Note that, in our example, our fictional customers answered like to the functional question and expect to the dysfunctional question. When we map this pair to the grid, we can see that the intersection of those two attributes is an E, as the yellow highlighted square indicates. Let’s examine what that means for our prioritized backlog.

E = Exciters. These are features that the customer did not expect and that truly differentiate a product from its competitors. They are difficult to identify, especially initially, because it can be difficult to come up with questions that elicit exciting features. For this reason, exciters tend to emerge and rise in priority as the project progresses and customer feedback begins.
- Customers receive great satisfaction from these features and are willing to pay a price premium to have them. For example, the iPod from Apple delighted customers with its intuitive ability to turn content to match the orientation of the screen. The absence of that feature would not have decreased satisfaction, however, because the customer wouldn’t have known to expect it.
- In our example, having GPS navigation would be an exciting feature. Exploring this feature, at least to the point of receiving customer feedback, should be a relatively high priority.
B = Baseline. These features must be in the product. They are the must-have, high-priority features.
- No matter how well these basic attributes are executed, the customer will remain neutral towards the product. A word processor, for example, must have “create document” and “save document” functions. They are the price of entry.
- If we had asked our customer group about seat belts, most people would answer that they expect a new car to come with seat belts and that they dislike it when it doesn’t. If we mapped those two answers on the grid, we would find that seat belts are a baseline (B), must-have feature.
L = Linear. Also known as performance requirements, linear features correlate directly to customer satisfaction. They fall just below baseline features in priority but must be balanced against their cost.
- Increased functionality or quality of execution increases customer satisfaction. Decreased functionality decreases satisfaction.
- Product price is often related to these attributes.
I = Indifferent. These features are least important to the customer and, as such, should be least important to the product. They will likely return little or no business value.

Table 1 also shows Q and R.

Q: Questionable – The question is probably incorrect or the answer is suspect.
R: Reverse – the combination of answers does not compute. Take the GPS navigation system – if someone answers that they expect it to be there but that they also like it if it is not, it’s an R answer.

If an answer pair is rated Q or R, you should investigate your questions or the answer pair more deeply.

Innovation Games

Innovation Games are tools to help you develop primary market research. With these tools, customers play “games” with the objective of generating feedback and input about a product or service. Luke Hohmann created and describes more than 12 of these games in his book, Innovation Games, which is a great addition to any library. My favorite two games to play to get a backlog prioritized are Prune the Product Tree and Buy a Feature, which Luke describes in this excerpt from that book (used with permission):

Prune the Product Tree

Gardeners prune trees to control their growth. Sometimes the pruning is artistic, and we end up with shrubs shaped like animals or interesting abstract shapes. Much of the time the pruning is designed to build a balanced tree that yields high quality fruit. The process isn’t about “cutting” – it is about “shaping.” Use this metaphor to help create the product your customers desire.

The Game

Start by drawing a large tree on a whiteboard or butcher paper or printing a graphic image of tree as a large format poster. Thick limbs represent major areas of functionality within your system. The edge of the tree—its outermost branches—represent the features available in the current release. Write potential new features on several index cards, ideally shaped as leaves. Ask your customers to place desired features around the tree, defining the next phase of its growth. Do they structure a tree that is growing in a balanced manner? Does one branch, perhaps a core feature of the product, get the bulk of the growth? Does an underutilized aspect of the tree become stronger? We know that the roots of a tree (your support and customer care infrastructure) need to extend at least as far as your canopy. Do yours?

Example layout for pruning a product tree

Product Tree –Hohmann

Why it Works

You and your customers both know that features vary in importance. So, we tend to want to put our efforts behind the most important features—those features that provide the greatest value to customers. Unfortunately, sometimes this means that we put too little effort behind the features that are needed to complete the product. The Prune the Product Tree game provides your customers with a way to provide input into the decision making process by looking at the set of features that comprise the product in a holistic manner.

Buy a Feature

Which feature will entice customers to purchase your product? Which feature will cause customers to upgrade? Which feature will make customers so happy that they’ll ignore or tolerate the features that they wish you would fix or remove?

Product planners endlessly debate these and other kinds of questions. Choosing the right set of features to add to a release often marks the difference between short-term failure or long-term success. Unfortunately, too many product planners make this choice without involving the people most affected by it—their customers. The Buy a Feature game improves the quality of this decision by asking your customers to help you make it.

Example layout for the game

Buy a Feature –Sterne

The Game

Create a list of potential features and provide each with a price. Just like in a real product, the price can be based on development costs, customer value, or something else. Although the price can be the actual cost you intend to charge for the feature, this is usually not required. Customers buy features that they want in the next release of your product using play money you give them. Make certain that some features are priced high enough that no one customer can buy them. Encourage customers to pool their money to buy especially important and/or expensive features. This will help motivate negotiations between customers as to which features are most important.

This game works best with four to seven customers in a group, so that you can create more opportunities for customers to pool their money through negotiating. Unlike the Product Box game, the Buy a Feature game is based on the list of features that are likely to be in your development roadmap.

Why it Works

Product planners often fall into the trap of thinking that customers have clearly defined product priorities. Some do. Most don’t. When presented with a set of options, many customers will simply say “I want them all.” and put the responsibility for prioritizing their requests on your shoulders. Alternatively, product managers often gather feature priorities by working with customers one-on-one and, in the process, and perhaps without even realizing it, once again take responsibility for prioritizing features. By engaging customers as a group and giving them a limited amount of resources, you give them the opportunity to prioritize their desires as a group. But that’s not where the magic lies. The magic lies in structuring the conversations so that your customers are negotiating among each other for specific features. It is this negotiation that enhances your understanding of what your customers really want.

Relative Weighting – Karl Weigers

Another excellent method for prioritization is Relative Weighting, which Karl Weigers introduced in 1999. This method not only provides a mechanism for prioritizing requirements based on user input and feedback but also includes the expert judgment of the team. Like Kano and Innovation Games, Relative Weighting allows the product owner to better gauge which features to implement and in what priority order.

The first step is to assign a weight to the relative benefit of a feature. A benefit is the advantage for users of having the feature in the final product. Next is to assign the relative penalty. The penalty is the consequence for users of not having the feature in the final product. (Assessing benefits and penalties accomplishes the same thing as the Kano method’s functional and dysfunctional form of the question.) The weights are arbitrary numbers that represent how your company values benefits and risks of features. It is very similar to how a teacher determines what weight to give homework, attendance, quizzes, and tests in determining the overall grade; it will vary from teacher to teacher. If you decide that the benefits outweigh the penalties, make the weight higher than the penalty by whatever ratio you see fit. If you decide that penalties outweigh benefits, adjust the weighting accordingly. In the example in table 2, we gave the benefit a relative weight of 2 and the penalty a relative weight of 1, meaning that the benefit will be a larger factor in determining the final priority.

In the same way, we determine the weight of cost percentage and risk percentage. In the following example, risk was not as much of a concern to the project, so cost percentage was given weight of 1 and risk percentage a weight of 0.5. (Note that, although benefit and penalty are weighted before the value percentage is calculated, cost and risk are weighted as percentages.) If you have a high-risk project, you might weight risk higher than cost.

Example Feature Table - Start

Now that the weights are set, we ask the users to rate each feature’s relative benefit and relative penalty. Each benefit and penalty is rated on a set scale—Weigers recommends 1-9, but you could choose a different scale as long as you’re consistent. The relative benefit is the value that the feature will deliver; relative penalty is the potential negative impact of not doing the feature.

For example, let’s say one possible feature is “Make the widget comply with Sarbanes-Oxley regulations.” There is virtually no relative benefit to users, but the relative penalty to the business is great - not doing it could put the company out of business. As such, we might see a score of 1 or 2 for the relative benefit and a score of 8 or 9 for the relative penalty.

In the following example, users rated the relative benefit of the feature “Query status of a vendor order” as a 5. They rated its relative penalty as a 3. To derive the total value of that feature, we multiply the two numbers by their relative weights and add them together:

(Benefit * Weight) + (Penalty * Weight) = Total Value

(5 *2) + (3*1) = 13

In this case, the total value for the feature is 13 points.

Example Feature Table - In Progress

When we get the total relative value and value percentage for the features, we move away from the users to get insight from the team. We ask the team to estimate the relative cost to implement each feature using the same scale. Karl Weigers explains, “Developers estimate the cost ratings based on factors such as the requirement complexity, the extent of user interface work required, the potential ability to reuse existing designs or code, and the levels of testing and documentation needed.”

After we estimate relative cost, we estimate relative risk. Again, Weigers says, “Developers estimate the relative degree of technical or other risk associated with each feature on a scale from 1 to 9. An estimate of 1 means you can program it in your sleep, while 9 indicates serious concerns about feasibility, the availability of staff with the needed expertise, or the use of unproven or unfamiliar tools and technologies. The spreadsheet will calculate the percentage of the total risk that comes from each feature.”

After we note the values for relative benefit, penalty, cost, and risk, we sum up each column. These totals will be used to calculate the percentages.

Total value percentage: Divide the sum value of the relative benefit and penalty by the total sum at the bottom. In the following example, this is 13 / 154 = 8%.
Relative cost percentage: Divide the relative cost value by the total relative cost sum at the bottom. In the following example, this is 2 / 42 = 4.8%.
Relative risk percentage: Divide the relative risk value by the total relative risk sum at the bottom. In the following example, this is 1 / 33 = 3%.
Priority: Divide the value percentage by (cost percentage * cost weight) + (risk percentage * risk weight). In the following example, this would be 8.4% / ((4.8% * 1) + (3% * 0.5). This gives the priority value (1.345).

After we obtain the priority values, we sort the priority column in descending order so that the highest priority items are at the top. As items are added to the product backlog or more information about a story emerges, we’ll need to reassess priority.

In the end, the sheet looks like this table:

Example Feature Table - Complete

By taking this approach, you can better understand the ranges that work for the team and for the stakeholders. It also helps to better ground discussions because it can be difficult to objectively factor in elements such as benefit, penalty, cost, and risk for each feature.

Weigers explains how to make the model more closely match your reality:

"Calibrate this model for your own use with a set of completed requirements or features from a previous product. Adjust the weighting factors until the calculated priority sequence agrees with your after-the-fact evaluation of how important the requirements in your test set really were. This model can also help you make trade-off decisions when you’re evaluating proposed new requirements. Estimate their priority using the model to tell you how they match up against existing requirements, so you can choose an appropriate implementation sequence. Any actions you can take to move requirements prioritization from the political arena into an objective and analytical one will improve the project’s ability to deliver the most important functionality in the most appropriate sequence."

Karl Weigers has written two books that go into more detail on relative weighting: Software Requirements, Second Edition and More About Software Requirements: Thorny Issues and Practical Advice.

Whether you use one of these methods or some other technique, remember that the product backlog is a living thing. You don’t just prioritize it once and then put it away—reprioritization is an expected part of maintaining a healthy backlog. To keep your project on track, you must constantly reassess the stories, their importance to the project, and how new information affects the backlog. You must take pains to keep your stakeholders and customers involved throughout the project. In addition, you must remember that an item’s estimate is essential for determining its priority. Don’t let your backlogs become stale and die. Invest the time and effort in nurturing and grooming your backlog—you’ll see the results not only the final product but also in your bottom line.

다음을 통해 공유