Wednesday, January 06, 2010

New Scrum Metric Proposal - "Hits and Misses"

If you want to compare Scrum Teams, compare them on who is achieving the Sprint Commitment (what I call "Hits and Misses"), not the Velocity.

Meaningful Metrics

One of the dangers with metrics, is that when we find a way to measure something, we tend to focus attention on it. Usually more attention than the metric is worth. So we like to oversimplify. What the "P/E ratio" say about the quality of a Company? What does "velocity" say about the quality of a Scrum Team? Both are pretty subjective numbers. The P/E ratio is tied to the stock price which is tied to the market's opinion of the value of the company, and markets are very emotionally driven. A Scrum team's velocity is tied to the User Story estimates which are subjective. The outcomes of comparing team velocities will most likely be negative, teams might inflate estimates or work overtime to increase their velocities.

Hits and Misses

I think it is very important to encourage the idea of Sprint Commitment. The team should take the Sprint Commitment very seriously. One way to encourage this is to measure how the team is succeeding in achieving the Sprint Commitment. I call this "Hits and Misses". Essentially, every Sprint Commitment achieved is counted as a "hit", every missed commitment is a "miss". If management is won't let go of the need to compare teams, I would give them the "Hits and Misses" metric instead of "Velocity". Of course there is a danger that teams will under commit. If this is happening, I might look into presenting both "Hits and Misses" and "Velocity". However, I rarely see teams under commit, the tendency is always to over commit.