This is the final article of three on effort estimating. In the first, I focused on definitions since many people use terms such as estimate and budget as synonyms when, in fact, they are very different. Here is a brief recap of the key definitions:
Effort. An expenditure of physical or mental effort on the part of a project team member. Effort is normally measured in terms of staff hours.
Estimate. An informed assessment of an uncertain event. Informed means that you have an identified basis for the estimate. Uncertain recognizes that multiple outcomes are possible.
Budget. A management metric that is derived from the estimate of the relevant work.
Baseline. A time-phased budget that has received all necessary approvals.
In the second article, I introduced the idea of three-point range estimates and covered the mechanics of how to prepare them with your team. I ended that article by asserting that three-point effort estimates usually take less time to prepare than traditional single-point effort estimates.
In this last article of the series, I’ll cover a potpourri of other effort estimating topics: understanding the expected result of effort estimating, converting effort estimates into budgets, dealing with poorly defined work, resolving conflicting effort estimates, converting effort into duration, maintaining your effort estimates, and what to do when management cuts your estimate in half.
Expected Result of Effort Estimating
The most important result from the effort estimating process is a labor budget that is accurate, i.e., one where difference between the budget and the actual result is acceptable. What’s acceptable will vary based upon the project, the domain, and the performing organization’s expectations. What's acceptable will also vary based upon whether you are comparing initial, order-of-magnitude efforts with full project actuals, or whether you are monitoring phase-by-phase.
I argue that an estimate is accurate if (a) the actual result for that work-item falls within the estimate’s range most of the time, and (b) the sum of the actual results is close to the sum of the expected values of the range estimates. In brief, we care about the total. We care about the individual activity results only as a means to an end.
Let me illustrate. Let’s say that we have a project with 100 activities. To keep it simple, let’s ignore scheduling concerns, and let’s further assume that each work-item has a three-point range estimate of 20, 25, and 30 hours of effort. The budget for each individual activity will be 25 hours (more on that later), and thus the budget for the project will be 2,500 hours.
If this project goes according to plan, most of the activities, probably around 70% of them, will be completed for an actual cost within an hour or two of their budgeted effort. One or two may take as many as 30 hours; one or two may take only 20. But the project total is likely to be very close to 2,500. Statistically, we have about a 95% probability that the project will finish within a range of ±50 hours. Not bad!
For comparison, let’s take a similar project with 100 activities. Only for this project, each activity has a range estimate of 10, 25, and 40 hours of effort. We will still budget this project for 2,500 hours, but ±50 hours has only about a 60% chance. If we want 95% accuracy, we need to go to ±120 hours.
Three-point range estimates help to support our goal of accuracy by encouraging everyone on the team to spend as much time and effort as they need to finish the work item correctly, and only the time and effort that they need. We hope to avoid a lot of dysfunctional behavior:
- Team members who use the full budget even if they don’t need to.
- Team members who cut corners to avoid exceeding the budget.
- Managers at all levels who waste time explaining minor variances.
- Managers who may be tempted to manipulate the reported actuals in order to make the estimates appear “better.”
And one final point: there are some circumstances when a single-point estimate is okay.
For example, we can safely predict that a one hour team meeting that includes seven people will require seven hours of effort. Is there some chance that only six people will show up? That it will take an hour-and-a-half? Sure, but this is likely to be such a small source of error that we can safely ignore it.
Converting Effort Estimates into Effort Budgets
While an estimate is a range, a budget is a single number. We can use some simple statistical techniques to convert our ranges into budget numbers. For this article, I’m going to ignore other budget preparation issues like unstable requirements and pre-defined budget maximums since these are more properly addressed through project risk management than through effort estimating.
The formulas for adding up ranges can be found in any basic statistics text. If you are willing to accept my contention that a triangular distribution is a good assumption, the basic process is straightforward:
-
Develop range estimates (low, likely, high; also called optimistic, most likely, and pessimistic) for each item that you are estimating. You can do this at whatever level of detail is appropriate for where you are in the project.
-
Calculate the expected value for each item by summing the three numbers and dividing by three. This assumes that the shape of the probability density function is a triangle.
-
Total the expected values. This is your base budget (i.e., it does not include any reserves or contingencies).
And yes, it is just that simple. Here’s a worked example for a small project with two deliverables and seven activities that produces a base budget of 290 hours:
Many of you are probably also familiar with the formula for the expected value that is used in the Program Evaluation and Review Technique (PERT). The PERT formula is different because PERT assumes a beta distribution. I prefer to assume a triangular distribution because it produces a more conservative result (a higher budget base) than a beta. Either assumption will work.
As an added bonus, most of today’s popular project management software packages have the capacity to deal with ranges in some way. If this feature is not built in, it is usually available via an add-on.
Dealing with Poorly Defined Work
In order to introduce the basic concepts of effort estimating, I limited our discussion to developing an estimate for a single, reasonably well-defined activity. Although this type of work may prevail in some project environments, most projects will have a significant amount of work to estimate that is not well-defined. In this section, I will cover some approaches to those situations.
No one on the team has experience with this kind of work. This is a two-edged sword.
Even if you can find an experienced resource to develop the estimate (e.g., a consultant or an article on the web), that individual may still have trouble estimating how much effort will be needed from an inexperienced person. Here are some ideas that should help:
-
Use a wider range estimate to reflect the greater uncertainty. Unless the unknown work represents a significant percentage of the total work, your overall project budget will not be greatly affected. Remember that we are willing to tolerate budget variances in individual work items as long as the project total is acceptable.
-
Verify your completion criteria. If the completion criteria are not clear, try redefining them until they are.
-
Break the work down into smaller units. Often, you will find that you do have experience with much of the work. The aspects where you don’t have experience can be handled with a wider range estimate.
-
Develop some experience. It may be possible to have one or two team members do a couple of “practice” activities as part of the project planning process. I used this approach quite successfully as part of a database conversion project several years ago.
We really have no idea of what’s required or we don’t know enough at this point to define the work. Neither of these two complaints is an effort estimating issue; they are both either a scoping issue or a risk management issue. They might also be a defensive response to a skeptical manager. If so, see below.
Resolving Conflicting Effort Estimates
Did you ever have two team members who differed widely in their opinions of the likely amount of effort required to complete a work-item? Range estimates can help, particularly if the individuals involved are inherently optimistic or pessimistic — if their ranges overlap, it is usually much easier to reach agreement.
But the root cause of such difference is most likely to be the presence of conflicting assumptions. For example, a team member who assumes that previous design work can be reused will produce a lower estimate than one who does not. By surfacing the conflicting assumptions, the team can discuss the alternatives and decide what to do.
One particularly pervasive assumption is the skill level of the person doing the work. Estimating is often done by more senior staff, and they tend to think in terms of “how much effort would I personally have to expend to complete this work.” This is likely to produce an estimate that is too low. At the same time, they may become aware of their tendency to underestimate and then overcompensate. I recommend that you remind your effort estimators to constantly assume an “average” resource, and to recognize that it will take 2-3 projects of comparing actuals to estimates for them to get good at knowing what “average” really is.
Converting Effort into Duration
I know that I am going to get a lot of flak from some of my colleagues about this next point. The technically correct answer about how to estimate duration is that you should do just that — look at the effort estimates and develop a three-point range estimate of the duration for each work item.
For example, with an effort estimate of 30, 40, and 80 hours that is to be performed by a single individual working fulltime on the project, I think it is reasonable to predict that this person may need more than two full weeks to get this item done if it ends up requiring 80 hours of effort.
In addition, since virtually all network analysis techniques (Critical Path, Critical Chain, PERT, etc.) underestimate the most likely duration of the project, many would argue that a Monte Carlo simulation is also a necessity. And I would agree … if you’ve got the budget.
If not, take the easy way out — take the effort budget (the expected value of the effort estimate) and convert that into a single-point duration budget based on the availability of the individual assigned.
Maintaining Your Estimates
If the work of the project increases via an approved change order, you will need to estimate the additional effort and update your budget as well.
If any of the assumptions that went into your effort estimates are changed, you should also update your estimates. In this case, you may not be able to modify the budget, but you should still re-estimate the work.
For example, if one of your most skilled staffers was supposed to work on Activity D, and due to circumstances beyond your control, Activity D is now being done by an intern from the local community college, you better reset everyone’s expectations.
Management Cuts It in Half
The first thing you have to do is find out why. Most such management actions are created by one of the following misunderstandings:
-
Your managers may not understand the difference between an estimate and a budget. Educate them. But in the nicest possible way if you want to keep your job.
-
Your manager may view your estimate as a negotiating position. Some managers assume that you have “padded” your estimate to provide negotiating room. Again, educate them - explain how the suggested cuts will either reduce quality or increase the risk of an overrun.
- Your manager’s cuts may be a negotiating position. Most good managers understand that a more difficult task (where the target is aggressive) can serve to motivate the project team. But even good managers often fail to appreciate that an impossible task (where the target is so aggressive that it is clearly impossible) is a powerful demotivator.
Summary
Underestimating (accidentally or intentionally) will drive you crazy. You will spend endless hours explaining why you are over budget and behind schedule. You will spend endless hours dealing with unhappy customers and stressed out team members.
Overestimating (intentionally) may not be much better. Your projects will not get approved because they will be seen as too expensive. You may even lose your job if your padding is viewed as unethical.
Three-point effort estimates will not get you into heaven. But they will make this life a lot more rewarding for everyone involved with your projects.
