Copyright © 2012 Corvus International Inc. All Rights Reserved
A hen is only an egg's way
of making another egg..
Samuel Butler (1835 - 1902)
Watts, on First
In the mid 90s I attended a presentation by the late Watts Humphrey where he remonstrated with a set of executives of a large US company for investing so heavily in metrics. The company had an SEI CMM Level 1 maturity rating which meant that their processes were "ad hoc" and therefore different for each project. Watts asserted that there was no point in careful measurement since all it would tell them was that their processes were different for each project...
...which they already knew.
The Five Core Metrics
According to the SEI and Larry Putnam Sr., there are only a few metrics we absolutely must collect. But even with these simple measures there are many issues with collecting them and what they mean.
- Size: this is simply a proxy for the knowledge content of a system. Larger, more complex systems contain more knowledge and are therefore more valuable. More knowledge requires more resources to acquire. System size metrics always measure the physical attributes of the substrate on which the knowledge is deposited. What we really want to measure is the knowledge, but there is no way to do that.
- Productivity: this is really attempting to measure the ability of the organization, team, or person to acquire knowledge, to learn. As any educator will tell you, this is a really hard thing to measure; but there are usable proxies for this too.
- Time: or duration. This is how long, in calendar terms, it will take an organization, team, or person to acquire the necessary knowledge. Measuring this is fraught with problems such as under-reporting overtime.
- Effort: or cost. This is dependent on how much you pay the people who are spending their time acquiring this knowledge. It suffers from many of the same issues as time measurement.
- Quality: or its evil twin, defects. Defects are simply things we don't or didn't know. Not only is defect measurement another loaded issue, even the definition of a defect can be, well, defective. [see Not Defect]
Intrinsic and Artificial Variability
When we measure things, there are
two kinds of variability that can creep into our
...So to summarize: in order to control something we have to measure it, but in order to measure it effectively, we need to exert some control....