In the first part of this series, I said I liked order-of-magnitude estimates. I also like targets in lieu of estimates. I’ll say more about how estimates can be useful in part 3.
In this part, I’ll discuss when I don’t like estimates.
I find estimates not useful under these conditions:
- When the people estimating are not the people doing the work.
- When managers use old estimates for valuing the work in the project portfolio.
- When management wants a single date instead of a date range or confidence level.
There are more possibilities for using estimates in not-so-hot ways. These are my “favorite” examples.
Let me take each of these in turn and explain how agile specifically helps these. That’s not because I think agile is the One and Only Answer. No, it’s because of the #noestimates discussion. I have used #noestimates in a staged-delivery project and on agile projects. I have not been able to do so on iterative or serial (waterfall/phase gate) projects. Of course, with my inch-pebble philosophy, I have almost always turned an iterative or serial project into some sort of incremental project.
People Estimate on Behalf of the Project Team
We each have some form of estimation bias. I have a pretty good idea of what it takes me to finish my work. When I pair with people, sometimes it takes longer as we learn how to work with each other. Sometimes, it takes much less time than we expected. I expect a superior product when I pair, and I don’t always know how long it will take us to deliver that product. (I pair-write with any number of people during the course of the year.) Even with that lack of knowledge, we can pair for a short time and project to a reasonable estimate. (Do a little work and then re-estimate.)
When people who are not part of the project team estimate on behalf of other people, they don’t know at least these things: what it will take the real project team to deliver, how the people will work together, and how/if/when the requirements will change. I have my estimation bias. You have yours. We might learn to agree if we work together. But, if we are “experts” of some sort, we don’t know what the team will encounter and how they will handle it.
I too often see experts ignore requirements risks and the potential for requirements changes. I don’t trust these kinds of software estimates.
Now, when you talk to me about construction, I might answer that we know more about construction. We have dollars per sq. foot for houses. We have dollars per road mile for roads. And, I live in Boston, the home of the Big Dig. Every time we remodeled/rebuilt our house, it came in at just 10% over the original number. We worked hard with the builder to manage that cost.
Those projects, including the Big Dig, were worth it.
How do we make software projects worth it? By delivering value as often as possible and asking these questions:
- Is there still risk to manage?
- Is there more value in the backlog?
- How much more do we want to invest?
Software is not a hard product. It is infinitely malleable. What we deliver on Monday can change the estimate for what we want to deliver on Tuesday, Wednesday and Thursday. We can’t do that with hard products.
When other people estimate, we can’t use what we learn by working together and what we have learned already about this domain. Agile helps this specifically, because we deliver often and can re-estimate the backlog if we need to do so. We understand more about the remaining risks because we deliver.
Managers Use (Old) Estimates for the Project Portfolio
I have seen managers use estimates to value projects in the project portfolio. I wrote a post about that years ago: Why Cost is the Wrong Question for Evaluating Projects in Your Project Portfolio.
Here’s the problem with old estimates. Estimates expire. Estimates are good for some time period. Not forever, but for some number of weeks. Depending on how you work, maybe the estimate is good for a couple of months. Estimates expire because things change: the team might change. The codebase and the requirements have certainly changed.
However, project cost is only one part of the equation. Value has to be another part when you think about the project portfolio. Otherwise, you fall prey to the Sunk Cost Fallacy.
You might say, “We use ROI (return on investment) as a way to value projects in the project portfolio.” Now you depend on two guesses: what it will take for you to complete the project and the sales/adoption rate for the release.
ROI is a surrogate measure of value. When I have measured the actuals (what it actually took us to finish the project and the actual revenue at three, six, nine and twelve months out, we almost always did not meet the projected ROI. And, because we chose that project with great-looking ROI, we incurred a cost of delay for other projects. “If we don’t release this project because we are doing something else, what is the effect on our revenue/adoption/etc?” (See Diving for Hidden Treasures to read about the different costs of delay.)
People often say, “These two projects are equal in terms of project cost. If I don’t use ROI, how can I decide between these projects?”
I have never seen this to be true, and it’s quite difficult to predict which project will be shorter. Here are some options:
- Use Cost of Delay as a way to value the projects in the project portfolio. See Diving for Hidden Treasures for ways to see Cost of Delay. See Manage Your Project Portfolio for many other ranking ideas for the project portfolio.
- Determine the first releasable deliverable of value for each project. How long will that take? If you do one project, release something, does that provide you enough revenue so you can go to the other project and release something there?
- Make all the deliverables small, so, if necessary, you could flow work from both projects through one team. The team can finish a feature/deliverable and move to the next one. I recommend using a kanban board and swarming over each feature so you get maximum throughput. Once the team has finished “enough” features, decide which project to spend more time on.
Agile helps the entire project portfolio problem because we can all see progress on an agile project: demos, product backlog burnup chart, and retrospective results. We know a lot more about what we finish and where we are headed. We can stop working on one project because we don’t leave work in an unfinished state.
Management Wants the Comfort of a Single Estimation Date
I supply a range of dates for my projects: possible, likely, pessimistic. I sometimes supply a confidence range. I have met many managers who do not want the reality of estimation. They want a single date: September 1, 2pm.
The problem is that an estimate is a guess. I can only know the exact duration or cost when I’m done with the project. I can get closer as we finish work, but I can’t know for sure months in advance. For a year-long project, I can guess as to which quarter/three month period. As we finish the project, I can spiral in on a date. By the last quarter, I can be within a couple of weeks of knowing.
Managers get paid the big bucks to manage the organization with assumptions, risks, and unknowns that we explain to them. When we work on projects, it’s our job to manage our risks and deliver value. The more value we deliver, the fewer unknowns our managers have.
Agile (and incremental approaches) help us manage those unknowns. Nothing is perfect, but they are better than other approaches.
I’ve worked with several managers who wanted one date. I gave them the pessimistic date. Sometimes, I provided the 90% confidence date. Even then, there were times we had more problems than we anticipated. Meeting that date became impossible.
A single-point estimate is something we like. Unfortunately, a single-point date is often wrong. Management wants it for any number of reasons.
If one of those reasons is assurance that the team can deliver, agile provides us numerous ways to get this result without a single-point estimate: set a target, see demos, see the product backlog burnup chart.
I have nothing against estimation when used properly. These are just three examples of improper estimate use. Estimates are guesses. In Part 3, I’ll talk when estimates might be useful.
(Sorry for the length of this post. I’ll stop writing because otherwise I’ll keep adding. Sigh.)
| Reference: | The Case for and Against Estimates, Part 2 from our JCG partner Johanna Rothman at the Managing Product Development blog. |
