Understanding a business using estimation modeling                               

An estimation model characterizes:

• The product or system a business is building in terms of its size, complexity, and required attributes of quality, documentation level, response time, etc.

• The organization or team that is building the system in terms of its capability/productivity, morale, processes, languages used, etc.

Specifically most estimation models require us to measure and/or predict these characteristics in some quantifiable way.  I am often asked to render an opinion, at extremely short notice, on an organization's estimates, commitments, plans, or status.  I have found that the discipline of building up estimation models is very helpful in understanding how a company works.

So this is what I did for an insurance client.

What is the "Cost of Risk"?                                                                       

For this client, I audited four of their projects and calculated that the likelihood of each project achieving its goals at the assigned levels of schedule, budget, and staff varied between 15% and 25%, averaging around 20%.  So I asked the client: "...what is the cost of risk on these projects?..."  Blank stares.  I changed tack and asked: "...what is the likelihood of these projects being successful?...". 

The managers thought for a moment and said  (quote) "um,... high?..."

They really had no idea what level of risk they were taking and assumed that the level of risk was non-existent of at least very low.  It wasn't.  In fact the chances of all four projects achieving their goals is approximately 0.2⁴  or about one chance in 600.  This makes going to Las Vegas and putting the department budget on one spin of the roulette wheel look like a really safe bet.

The Insurance Business                                                                       

I'm not knocking the insurance business in any way, but it is rather ironic that a business whose primary skill and function is the precise calculation of risk in pretty much everything it does, does not apply this to software projects. 

What is the "Cost of Risk"? (again)                                                       

Estimates are "inaccurate"¹ because there are things we don't know about the project.  Clearly to make an estimate "accurate" we'd have to find out those things we don't know.  To find out those things will take us time, effort, and budget.  And that is primarily where the cost of risk comes from.

Cost at Risk and Return at Risk                                                                                       

In fact, we should not be quoting the cost of a project at all we should be quoting the COST AT RISK of the project.  The Cost at Risk is the calculated cost of the projects plus the calculated cost of risk.  When a decision is made to fund a project at a lower probability of achieving the allocated budget, usually by reducing the budget, schedule, staff or some combination of these, the "cost" of the project may go down, but the cost of risk of the project will go up.

I am working on a tool that will perform this calculation and it is a topic to which I will return. 

Watch this space.

¹ This actually a meaningless statement but one we all use.  To see why see: Ten Unmyths of Estimation