Thoughts on project risk

by Robert Buttrick

One of the key note speakers at the Gartner PPM Summit in London this year was Professor Bent Flyvberg of Oxford University. His talk was about “Why your IT project might be risker than you think”.  In this he summarised the outcomes of some research covering over 4,000 projects, from a range of industries, with differing durations and sizes.  The first stunning finding was that, contrary to common belief, IT projects were no more risky than engineering projects – the median for cost overrun is about the same.

“Good”, you might think. However that may be a premature conclusion to jump to. What they also found was that if an IT project goes out of control, then it really goes awry – big time. Further, IT projects were 100 times more likely to go out of control. It’s all to do the statistical distribution and 'outliers'. Now this sounds more like the story our guts were telling us . . . . but worse.  These outliers are termed “black swans”, on the basis that you rarely see black swans (except if you live in New Zealand, of course).

Now all sectors have black swan projects, but the IT sector really seems to have some issues. Despite the projects being far shorter in duration, the IT black swan projects have significantly higher costs over-runs.  This led the research team to look at a few more hypotheses:

Bigger projects are riskier than smaller ones – WRONG, they found that the risk of 'cost black swans' decreased with project size.

Longer projects are riskier than shorter ones – TRUE, longer projects are riskier, especially after 3 years

Lack of benefits management is the single most important deficit in IT project performance management – TRUE, benefits management significantly reduces risk.

Using Agile methods lowers project risks – TRUE .  . . but only for schedule risks, not for costs or benefit.

The problem with the above is that the challenge is how to deal with the black swans, bearing in mind the significant impact they have. One approach is spot them early, but when is that? When is a black swan hatched.  Their research showed four situations:

  1. As soon as a vendor has been contracted – watch those contract claims go through the roof! About a fifth are like this.

  2. When the system is being specified, after which the costs stabilise. This covers about a half of the black swan projects.

  3. Just under 10% look great right up until actual development starts and then the escalations come in.

  4. That leaves just under a fifth, which are actually starved to death.

The best early warning indicator is people insisting a project is really unique. 'Unique' and black swan go together very well. Further the research indicates that the best time to look at a project is 15% of the way through; they believe that by this time the die is cast and a review may catch the renegade project before it does too much damage to the organisation.

Here is some advice and references if you want to take this further:

  • benchmark against your previous projects, your industry and best practice;

  • know your own uncertainty and risks;

  • improve resource allocation and eliminate knock on effects;

  • scrutinize your plans;

  • know how likely estimated costs, benefits and schedules are to actually materialise;

  • quantify project viability under different scenarios;

  • reduce the front end bias of your projects;

  • identify and eliminate delusion, deception and black swan blindness;

  • quantify unknown unknowns;

  • think carefully about reducing scale;

  • reduce technical and social complexity;

  • learn from 'master builders' who have a proven track record.

 

References:

  1. From Nobel Prize to Project management getting risks right. PMJ, 37(3) pp 5-15
  2. “Why your IT project might be riskier than you think” HBR, 89 (9) pp 23-25

© Robert Buttrick

http://www.projectworkout.com

SHARE THIS PAGE

Please consider allowing cookies to be able to share this page on social media sites.

Change cookie settings
No history has been recorded.
Back to top