Pop quiz: What do earned value, burn-down charts, and coverage reports have in common?
They are all reporting a metric, sure. But you could have gotten that from the title. Let’s dig deeper.
Let’s start with earned value. Earned value is a project tracking system, where you get value points as you’re getting along the project. That means, if you’re 40% done with the task, you get, let’s say, 40 points.
This is pre-agile thinking, assuming that we know everything about the task, and nothing can’t change on the way, and therefore those 40 points mean something. We know that we can have a project 90% done, running for 3 years without a single demo to the customer. From experience, the feedback will change everything. But if we believe the metrics, we’re on track.
Burn-down charts measure how many story points we’ve got left in our sprint. It can forecast if we’re on track or how many story points we’ll complete. In fact, it assumes that all stories are pretty much the same size. First, the forecast may not tell the true story, if for example the big story doesn’t get completed. And somehow the conclusion is that we need to improve the estimation, rather than cut the story to smaller pieces.
Test coverage is another misleading metric. You can have a 100% covered non-working code. You can show increment in coverage on important, risky code, and you can also show drop in safe code. Or you can do the other way around, and get the same numbers, with the misplaced confidence.
These metrics, and others like them have a few things in common.
- They are easy to measure and track
- They already worked for someone else
- They are simple enough to understand and misinterpret
- Once we start to measure them we forget the “what if” scenarios, and return to their “one true meaning”
In our ever going search for simplicity, metrics help us minimize a whole uncertain world into a number. We like that. Not only we don’t need to tire our mind with different scenarios, numbers always come with the “engineering” halo. And we can trust those engineers, right?
I recently heard that we seem to forget that the I in KPI is indicator. Meaning, it points in some direction, but can also take us off course, if we don’t look around, and understand the environment.
Metrics are over-simplification. We should use them like that.
They may have a halo, but they can also be pretty devilish.
| Reference: | Metrics: Good VS Evil from our JCG partner Gil Zilberfeld at the Geek Out of Water blog. |
