Estimating an issue

In Scrum, there is a distinction between estimation and tracking. Estimation is typically performed against Primary Backlog Items (PBIs, usually stories), and is used to work out how long portions of the backlog might take to be delivered. Tracking refers to monitoring the progress of a sprint, to be sure that all stories included in the sprint will be delivered.

Tracking is often performed by breaking down stories into tasks, and applying hour estimates to them during sprint planning, then monitoring the remaining time in a burndown during the sprint.

In traditional development environments, estimation is done this way:

1. A team estimates items in 'man-hours' – and these estimates are assumed to be accurate.
2. The team then calculates the total number of man-hours for the backlog of a project.
3. The team then divides the total number of man-hours by the number of people on the team, and the man-hours in a week. This becomes the forecast date for the project.

These estimates are often inaccurate because they don't consider the following:

• The natural estimation characteristics of the team – meaning, over- and under-estimation are not considered
• Unexpected interruptions during the man-hours allocated to the items
• The performance of the team members themselves over time

When the estimates become inaccurate, the team then exerts time and effort in trying to 'force' the estimates to be accurate. This makes the man-hour approach difficult — if not impossible.

In the Scrum world, teams don't try to achieve estimation accuracy. Instead, they aim to achieve 'reliable velocity'.

Velocity is a measure of the number of estimation units that a team tends to complete from sprint to sprint. After their first few sprints, most teams will achieve a reasonably consistent velocity. Armed with velocity and estimates on the PBIs in the backlog, teams can predict more accurately how long portions of the backlog will take to complete.

The key is, the estimation unit doesn't matter – as long as it becomes reasonably predictable from sprint to sprint. For example, teams can use 'ideal hour' estimates, but it's neither necessary or expected that those hours will have any relationship to elapsed time. If a team has a 'man-hour' capacity of 120h in each sprint but a velocity of 60h, that makes no difference because you can still use the 60h velocity to estimate the number of sprints that portions of the backlog will take to complete — and therefore, the elapsed time.

Many people then start wondering where 'the other 60 hours' went, thereby implying that there is something wrong with team productivity. But that's usually got nothing to do with it: a team's estimates merely represent their view of how hard items will be, taking into consideration the team's natural behavior (such as over- and under-estimation), as well as organizational overhead, etc. The velocity is all that matters from a planning perspective. 

Since the units are not related to time, most teams now choose to use story points as their estimation unit. A story point is an arbitrary number that measures the complexity of one story relative to others. In effect, story points clearly break the mental link with time.

For a team's velocity to reach a stable state, the team must estimate each backlog item with the same level of accuracy. At the risk of repeating the obvious, the goal of velocity is to be able to look at a backlog of not particularly well-understood stories, and understand how many sprints it will take to complete. This requires a similar level of uncertainty for all of the estimates in the backlog.

There is a counter-intuitive implication here — that teams should estimate each item once, and not change that estimate even if they discover new information about the item that makes them feel their Original Estimate was wrong. If the team were to go ahead and update estimates, this 'discovery of new information' will happen regularly. This leads to the backlog having some items that have higher accuracy, but most that don't. This would pollute velocity because sprints with a larger percentage of high accuracy estimates will complete a different number of units compared to those with a lower percentage of high accuracy estimates. As a result, the velocity could not be used for its primary purpose — that is, for estimating the number of sprints it will take for a team to complete a set of not-well-understood stories in the backlog. Therefore, it's critical to use the first estimates so that the team's velocity realistically represents their ability to complete a certain number of units of not-well-understood work far ahead into the future.

Consider the following scenario:

• Issue X has an Original Estimate of 5 days.
• Before the next sprint is planned, the team realizes that the Original Estimate was too optimistic, and that the issue actually takes 15 days. 

Some people would argue that using the Original Estimate will endanger the sprint's success, because the team will take in what they think is 5 days of work into the next sprint when it's actually 15 days of work.

However, the inaccurate estimate of 5 days is unlikely to be an isolated occurrence. In fact, the estimates are always going to be wrong (some very little, some wildly so). This will often be discovered after the sprint has started rather than before. As long as the team estimates the same way across the whole backlog, this will work itself out over time. For example, if they always underestimate, they may find that for a 10-day sprint with 4 team members, they can only really commit to 20 days of their estimation unit. If they have established a stable velocity, then this has no effect. From a planning perspective, we can still reliably estimate how much work we'll get done in upcoming Sprints.

When the team is about to start a sprint, they can use the velocity as an indication of items from the backlog that they can realistically complete. The velocity here is based on the number of items they have successfully completed in the past. However, some people may question how this can be right when the Original Estimates won't include information about work that may have already been done, or information about how hard a particular item of work is.

As an example, consider the following scenario: 

• An issue has an Original Estimate of 10 days.
• The team works 5 days on the issue in the current sprint.
• The team discovers a bad bug somewhere else in the project, and they decide that fixing that bug in the current sprint is far more important than completing issue X as planned.
• The sprint gets finished, and the issue returns to the backlog.

In the next sprint, the team would be tempted to update the estimate for the issue to 5 days, and use that to make their decision whether or not to include it in the sprint. The implication is that they might not include enough work in the next sprint if they used the issue's Original Estimate of 10 days. However, the reason that the task was not completed previously is because of unplanned work — and it's unrealistic to assume that this won't happen again in the future, perhaps even in the next sprint. Thus, the 10-day estimate is a realistic number to use in the absence of certainty. As a result, the cost of the unplanned work that may happen is eventually accounted for in the Original Estimate. Even if the work does turn out to be insufficient for the next sprint, the team will correct that by dragging more work into the sprint.

In the same example, consider if this were the only issue in that sprint and will be the only issue in the next. If the issue is completed in the second sprint, and we use the Remaining Estimate, then the velocity will be (0d + 5d) / 2 = 2.5d. However, the team can clearly complete more work than that in future sprints. If we use the Original Estimate, then the velocity will be (0d + 10d) / 2 = 5d. The use of the Original Estimate accounts for the fact that the team cannot commit to 10d in every sprint because unplanned work will likely make that impossible. It also realistically accounts for the fact that unplanned work will not happen in every sprint.

Many teams break down stories into sub-tasks shortly before the sprint begins so they can use the stories for tracking. This raises the possibility of using the sum of the estimates on the sub-tasks as a way to decide which issues to commit to in the sprint (and potentially for velocity). 

As described above, tracking is really a separate process from estimation and velocity. The estimates that are applied to sub-tasks clearly have higher accuracy than those that were originally applied to the story. Using them for velocity would cause the velocity to have both high and low accuracy estimates, making it unusable for looking further out in the backlog where stories have only low accuracy estimates.

In addition, only items at the top of the backlog are likely to have been broken into tasks. Using task estimates for velocity means that the velocity value could only predict the time to complete the backlog up to the last story that has been broken into tasks.

Lastly, using sub-task roll-up to decide sprint commitment is risky because, unlike velocity value, it doesn't consider the overhead of unplanned work and interruptions.

More and more industry leaders are moving away from hour estimates, and are now using the story point approach. This makes sense because in a sprint, the main questions to be answered are:

• How much work can we realistically commit to completing this sprint?
• How long will this part of the backlog take to deliver?

The story point approach based on original estimates can deliver the answers to these questions without the anxiety around 'accuracy' that teams feel when asked to estimate in hours.

The JIRA Software team itself uses the approach described in this article, and has established a reliable velocity that we use to plan work months in advance — even when new work has been encountered during those months. We recommend this approach because while it is sometimes counter-intuitive, it is also powerful, fast, and simple. 

All of that said, one of the key precepts of agile is finding the way that works for you. So JIRA Software does support the alternatives described above, including the use of remaining estimates for sprint commitment, hours for estimation, and hour estimates on sub-tasks.