Estimating
By Mitch Lacey. Owner, Mitch Lacey & Associates, Inc, a consulting firm specializing in agile and scrum adoptions and improvements.
January 2012
Mitch Lacey discusses the difficulty surrounding software project estimation, and provides tips and tricks for using two agile software estimation techniques when estimating projects.
Applies To
Application Lifecycle Management; Team Foundation Server
Contents
Introduction
Why Estimation Is Hard
Estimation Techniques
Story Points as Unit of Measure
Planning Poker
Wall Estimation
Estimation
Prioritization
Conclusion
Estimating work that is creative and unpredictable is just plain hard. Choosing a way to do it can be equally taxing. Fred Brooks says it best: “It is very difficult to make a vigorous, plausible, and job-risking defense of an estimate that is derived by no quantitative method, supported by little data and certified chiefly by the hunches of the managers.”
Yet we are asked to give estimates for our software projects up front and early—and despite all our efforts to remind management that these estimates are rough, all too often our initial estimates turn into commitments.
In this article, I will show you why it is challenging to estimate projects up front, how agile software estimation techniques help, and how to estimate your product backlog using planning poker and story points.
Why Estimation Is Hard
In most projects, we are asked to estimate up front. To understand why this is such a problem, we must examine the Cone of Uncertainty, which Barry Boehm introduced to us in 1981 and Steve McConnell re-introduced in 1997 in his book Software Project Survival Guide.
.png "Cone of Uncertainty")
The cone demonstrates that we have the most uncertainty at the beginning of any project (a variance of 4x to 0.25x in range). This variance means that what we estimate to be a one-year project could actually end up taking anywhere from 3 to 48 months. The beginning of any project is the time when we are the least certain about the project, yet it is also when we are asked to deliver estimates that are very precise.
In agile, we try to move from uncertainty to certainty in as short a cycle as possible. This is accomplished by maximizing early learning about the system and how it should be designed. To do this, we create a single path through the system, a complete and working story. We use this to flush out design and requirement assumptions early, which allows us to move to certainty much more quickly and with much more confidence.
Estimation Techniques
There are a wide variety of valid estimation techniques, including counting, expert judgment (individual and group), decomposition, analogy, proxy estimation, planning poker, and wall estimation. We can also use tools such as Cocomo II. All of these techniques require that we choose a unit of estimation—hours, days, weeks, months, ideal days, T-shirt sizing, points—or all of them. If we don’t understand the unit, the estimate means nothing. Considering all of this, it is no wonder why we struggle with estimation.
Although agile teams gravitate toward a certain flavor of estimation units and techniques to estimate the product backlog (story points and planning poker), your team should feel free to use whatever method works best for its needs. In my experience, expressing estimates in terms of story points, using either planning poker or wall estimation, yields the best results. In the following paragraphs, you will learn about story points as a unit of measure and two estimation techniques that I prefer: planning poker and wall estimation.
Story Points as Unit of Measure
Mike Cohn summarizes story points best: “Story points are a unit of measure for expressing the overall size of a user story, feature or other piece of work. ” Story points tell us how big a story is, relative to others, either in terms of size or complexity. Mike often refers to “dog points” when helping teams understand the concept of relative sizing. A 2-point (small) dog would be a Chihuahua. A 13-point (big) dog would be a Great Dane. With those two guides in mind, it’s fairly easy to size the other dog breeds relative to a Chihuahua or Great Dane. A Beagle, which is about twice as big as a Chihuahua, might be a 5. A Labrador, which is bigger than a Beagle but smaller than a Great Dane, might be an 8.
When you are first learning to use story points, your team will need to establish your own fixed comparison points. To do this, choose a story from your product backlog you can all agree is small (either in terms of size or complexity) and one you all agree is huge. I like having my small story be a two-point story because, if I need to go smaller (say I discover a Toy Chihuahua), I can. If I limit my smallest known story to a one-point story and I need to go smaller, I’m in trouble. The other stories can then be sized relative to these.
When it comes to choosing numbers to represent these sizes, I find the Fibonacci sequence to work the best. Fibonacci is the sum of the previous two numbers. So 1 and 2, the next is 3. 3 and 2, the next is 5. 5 and 3, the next is 8, and so forth. I prefer Fibonacci over, say, T-shirt sizing or growing exponentially (4/8/16/32/64/128/256, etc.) because we humans are good at base ten. When we get out of that range, even if we are in it with, say, xs, s, m, l, xl—it becomes confusing. Fibonacci numbers are simple, easily understood, and provide enough accuracy to get us to the goal—providing relative estimates. You can choose a different set of numbers, but remember that the important thing is to be consistent.
Story points are relative values, not fixed. There is no direct correlation between hours and points. For example, we cannot say with any degree of certainty that a two-point story is equal to 12.2 hours because stories in the two-point range will vary greatly in how many actual hours it takes to complete them. Similarly, you cannot compare one team’s story points with another’s with any degree of certainty. Story points are created by and are specific to the team that estimated them, will likely include a degree of complexity that is understood only by the team, and are not absolute.
Planning Poker
After you’ve chosen your unit of measurement and established your scale, it’s time to estimate. Most Agile teams use planning poker to estimate the relative size of the stories. It is popular with agile teams because it is an objective measure that includes several subjective estimation techniques, including analogy and expert judgment. The key to planning poker is participation. Everyone on the team needs to participate—yes, everyone. Functional testers will estimate development tasks and vice versa. Functional project managers might also estimate development tasks. Doing this ensures that your objective numbers include as much subjective estimation as possible.
Start with a set of planning poker cards. Planning poker cards can easily be made with index cards, rigged through the use of playing cards, or purchased—Visual Studio even makes them. Each card has one of the numbers in your chosen range of story points (1, 2, 5, 8, 13, etc.). Every participant is dealt a “hand” that contains the full range of available story points.
.jpg "Example planning poker cards")
After the cards are distributed, the game begins.
The ScrumMaster presents the top item in the product backlog to the team.
The team discusses what the story is.
The product owner clarifies questions, assumptions, and unknowns—as well as acceptance criteria.
Each team member privately decides how big this story is relative to a reference story, a series of reference stories, or all of the stories on the product backlog.
At the count of three, everyone shows his or her chosen card simultaneously.
If everyone played the same card, the team can log the estimate and move on to the next story.
If there is a wide variance (for instance, the displayed numbers range from one to eight), then the team spends time discussing the story. To focus the discussion, have the low bidder and the high bidder both explain their reasoning for their estimates. The conversation is valuable here, not the number, because that’s where the learning occurs and any assumptions are uncovered. After a brief 30-second to one-minute discussion, the team repeats steps 4 and 5. This continues until the team agrees on an estimate for the story.
This seems relatively straightforward, but it is important to understand some basic rules. First, if the team does not all come to some sort of agreement, you should not move on. For example, let’s say that there is one person on the team who plays an eight but the others all choose five. If the meeting facilitator says, “Close enough. We’re going with the five on this one; move on,” what will the person do who had the eight? Based on my experience, that person will go with whatever the team decides but will stop participating fully. Planning may go more quickly that way, but you’ve lost something valuable. Not only does this person lose out on understanding the work, the team loses one member’s input and perspective. Plus, it is OK to disagree. Valuable understanding comes from discussions over why one person chose a higher number than everyone else. If you find yourself in a stalemate, have the facilitator try the “Fist to Five” technique. It works wonders to get meetings moving along without alienating any of the participants.
Because planning poker expresses estimates in points, it is ideally suited for estimating the product backlog. The sprint backlog, however, should be estimated in hours. That being said, planning poker can and has been used successfully to estimate sprint backlogs; the numbers on the card, though, become hours, rather than points. So, simple rule –
Product backlog estimates are in points.
Sprint backlog estimates are in hours.
You can use planning poker at the beginning of any project and throughout its lifecycle as new information reveals itself, priorities change, and clarity surfaces.
Wall Estimation
Planning poker is a fantastic tool for estimating user stories, but it would take an inordinate amount of time to estimate hundreds of stories, one at a time, using planning poker. If you have a raw backlog filled with hundreds of stories that have not been estimated or prioritized, you’re going to need a faster way to estimate.
Wall Estimation is designed to allow teams to eliminate discussions of 2 versus 3 and 5 versus 8 and instead group things in a purely relative manner along a continuum, at least initially. It also allows stakeholders to give a general prioritization to a large group of stories without getting hung up on whether one story is slightly more important than another.
To do Wall Estimation, you must first print your user stories on cards. Then gather your team and stakeholders in a room with a big empty wall (about 14 feet long by 8-10 feet high); Understand two things about the wall:
Height determines priority. Stories at the top are higher; stories at the bottom are lower. A story’s priority can be based on ROI, business value, or something as simple as “it’s just important, and I don’t know why.”
Width is reserved for size. Stories on the left are smaller; stories on the right are bigger. (You can reverse this and move from right to left if you’re in, say, Japan and it’s more logical.) The important thing is to envision a line going horizontally and one going vertically. Team members and stakeholders should ask themselves, where, relative to the other stories, does this one fit?
The team will use the wall to size all of the stories. The stakeholders will use the wall to prioritize stories. As with planning poker, we’re using relative sizing, but instead of using two reference stories for comparison, the wall becomes the constant. Small story? Move to the left. Big story? Move to the right. Important story? Place it high. A story that we can live without for now? Place it low.
Although the stakeholders do not have to be there while the stories are being estimated, the team does need to be in the room while the stories are being prioritized. The ScrumMaster and product owner must attend both the estimation and prioritization activities.
Now, if you already have an estimated backlog, you can just do the prioritization section of this exercise. If your product owners and stakeholders have already given you a prioritized backlog, you can just do the estimation section of this exercise. (Your product owner will likely want to revisit the prioritization after the estimates are done. After all, cost has a big impact on priority.) Let’s look in detail at how this would work, beginning with the team’s role.
Estimation
Give the team the raw product backlog, and start with estimation. Instruct the team that the far left of the wall should hold the smallest possible stories and the far right should contain the biggest possible stories, without regard to numbers. The team puts stories somewhere on the wall based on those two poles. The advantage to doing it this way is that there is no preconceived notion of what a two-point or three-point story is; it’s truly relative based on how big the wall is, which is why a really big wall comes in handy.
If your team is having difficulty doing this, you can give more structure to the wall by providing additional reference stories ranging from 1 to 8 points. Don’t worry about creating a bigger reference story; anything bigger will usually be broken down as it rises in priority. After the team has identified the five stories, place them on the wall in a location that correlates with their sizes (again moving from left to right). Leave some room on the right side of the wall for stories bigger than eight points. Place these stories on the wall, and instruct the team to place the remaining stories on the wall relative to these reference stories, keeping in mind that smaller stories go to the left and bigger stories go to the right.
With the stories on the wall, have the team identify the logical breaks between story sizes. Tape a vertical line between groups of stories to illustrate these breaks. Soon you’ll have a wall that looks something like the wall shown here. Everything in the first group might be a 2; everything in the second, a 3; everything in the third group, a 5; and everything in the last group, an 8. The numbers don’t matter as much as the fact that all the stories are now estimated relative to each other.
.png "Example Wall Estimation - Relative Sort")
Now that you’ve estimated your stories, you’ll need to involve your stakeholders so that you can assign the stories a priority.
Prioritization
Although your customers and stakeholders will want to know how big a story is to help them determine its priority, they’ll be much more focused on finding the stories that relate to them and making sure those stories get done. Expect your stakeholders to disagree about priority—your product owner will use this information to help decide the ultimate priority.
Explain the wall to the stakeholders. Tell them that the cards on the wall reflect all of the features they want to see in the final product. Explain that the team has already estimated each story and that they can determine the point estimate for a story based on which column it is in on the wall. Remind everyone that the team members are not active participants in prioritization. They are there to observe, take notes on behavior, interactions, and the reasons why certain stories are rising or falling in priority. They can also answer any questions from the stakeholders if needed. If the team couldn’t size one or more stories with confidence because it required answers from a particular stakeholder, the team can also ask questions about those stories, as time allows.
Ask the stakeholders to help determine the relative priority of all of these stories by moving these stories up or down inside the taped columns. Remind them that the higher up a story is on the wall, the higher its priority to the business. Set the following rules:
If you place a story at the top, be prepared to justify the placement.
You may ask each other why one story is more important than another. Feel free to ask each other, “Who moved this one down (or up)?” or to say aloud, “I think this one needs to move. Who wants to disagree?” This enables conversation between the interested parties, without facilitation.
If you move a story lower on the wall than someone else did, mark it with a colored dot to alert us.
The biggest benefit to prioritizing as a group is that all the stakeholders can better understand the priorities of various stories. If a discussion goes on too long without resolution, the product owner should collect the card, identify the two stakeholders who cannot agree, and make a note to meet with them privately later.
The exercise could take 2-6 hours, depending on the number of stories and the number of stakeholders. When you are finished, the wall will look something like the picture shown below.
.png "Example Wall Estimation - Priority Sort")
Your wall will break down roughly into four quadrants. The stories in the top left are high priority and small so they’ll end up in the top of the product backlog. The stories in the top right are high priority but are also large. These stories should be broken down soon so they can be brought into upcoming sprints.
.png "Wall Estimation - Four Quadrants")
The lower left quadrant is made up of small stories that are lower in priority. They will likely fall to the bottom of the backlog. The lower right quadrant is filled with large stories that are also lower in priority. These stories are your epics or themes. They’ll eventually need to be broken down into smaller, more manageable stories but not until they rise in priority.
Spend some time looking at the wall as a whole with the group. If a story is in the wrong quadrant, move it. If a high-priority story must be broken down and time allows, do it while everyone is in the room.
At the end of wall estimation, you’ll have the start of a release plan. If you know the team’s historical velocity, you can even supply a rough range of which stories in the upper-left quadrant will be finished.
Estimation is hard because there is so much uncertainty at the beginning of a project. Product Owners and agile project managers try to maximize value early learning by having conversations with their product owners and stakeholders, producing working software, and integrating feedback about that software to get to a releasable state. But even agile projects must provide some estimate of when a set of features will be ready for release.
The two estimation techniques that I recommend are planning poker (appropriate for smaller sets of stories) or wall estimation (good for managing large raw product backlogs). Either will give you the data you need to begin forming a plan, which is the ultimate goal of estimation.
See Also
Concepts
Groom and estimate the backlog