How can project managers reduce risk in an uncertain world?

In the current climate, risk management is at the top of every project manager’s agenda. COVID-19 was a low-likelihood, high-impact catastrophe. The end of the Brexit transition has disrupted supply chains. In the next few years, the cost of dealing with COVID-19 will have to be paid back. The UK government has outlined a plan of tightened spending and increased taxes, other governments will follow suit.

When money is tight and uncertainty is high, how we spend every single pound counts.

As project professionals we need to ask ourselves, how good are we at managing risk, and can rare and unknown risks even be managed? The answer is that they can, but we need to get better at protecting our projects from these surprises. Reference class forecasting (RCF) is crucial to building resilient projects, as it is the most accurate methodology to improve forecasts.

How can you manage rare and unknown risks?

First, we must get better at managing risk to the point that nothing should be allowed to fail under conditions that are predictable. That way, when the exceptional happens, organisations are more resilient.

Second, we need to understand that most risks, even rare or unknown risks, can be managed. It’s true that bottom-up forecasting methods are not useful to manage rare or unknown risks. But RCF, which is a top-down method, is.

This means that project failures should be rare. The problem is that the reverse is commonplace - project failure is a very common outcome. Only 0.5 per cent of projects are delivered on time, on budget and with the benefits expected^[1].

Why do projects fail so often?

Extensive research by Professor Bent Flyvbjerg, Dr Alexander Budzier and colleagues has proven the Iron Law of Projects; projects are 'over budget, over time, under benefits, over and over again'^[2]. 

You can listen to Professor Bent Flyvberg share his knowledge, expertise and practical advice on how projects can be better planned and managed on the APM Podcast.

This is as true for infrastructure or IT projects as it is for the largest event on the planet - the Olympic Games. Their research finds that more than 70 per cent of transport projects exceed their budget and nearly 2 in 3 experience delays. Fifteen of the 19 Games from 1960-2016 overspent by more than 50 per cent, and 10 cost more than double their original budget.

So, what’s going on? These kinds of failures can hardly be blamed on unpredictable events when you realise that they happen to most projects. The answer is that project cost and schedule are systematically underestimated and benefits are over-estimated.

The root causes of erroneous estimations are optimism bias and political bias. Optimism bias is an unintentional tendency; we tend to overestimate our abilities and underestimate the likelihood of problems arising. Political bias is often deliberate misrepresentation to achieve political and economic aims, for example when we bid to win. These biases lead to underestimated cost, schedules and risks, simultaneously causing benefits to be overestimated.

We are all prone to these biases, including experts. Even knowing that decisions are biased, doesn’t de-bias our planning. This explains why projects fail so often.

How does reference class forecasting manage risk better?

Based on a Nobel Prize-winning theory^[3] adapted by Flyvbjerg^[4], RCF has been proven to be the most accurate method for project forecasting.

The fundamental advantage of RCF is that it bypasses psychological and political biases that result in the planning fallacy. It does this by taking an ‘outside view’, rather than just considering internal evidence and experience (an ‘inside view’).

RCF works by analysing a group of finished projects and uses those data points to build a risk forecast. By incorporating all effects on performance of past projects, including unexpected events and rare disasters, RCF can provide for ‘unknown unknowns’ and exceptional events. Depending on the dataset, it’s possible to learn from history stretching back to the 1930s.

Forecasts can be made for performance, as well as for risk. These forecasts can cover any quantifiable project factor but are most frequently used for cost, schedule and benefits. The three steps of RCF are:

1. identify a relevant reference class of past projects,
2. establish a probability distribution for the assembled reference class, and
3. determine the most likely outcome for the project being appraised.

An example of a probability distribution is presented below, adapted from figure 1.3 of Reference Class Forecasting guidelines. Assuming our project is like those in the reference class, our project will sit in the middle of the distribution. This gives a 50 per cent probability (P50) that our project will overrun by 17 per cent or less. That means a 17 per cent budget uplift is needed to be 50 per cent sure that our project will not overrun. If this chance of overrun is too high for our risk appetite, we can calculate a more conservative budget uplift of 57 per cent. That will provide us with 80 per cent certainty (P80) that our project will not overrun.

 

Where can I learn more?

Given the failure of most projects, you may be forgiven for assuming that RCF is an advanced technique, with limited practical application and restricted to academics. But that’s not true.

RCF can be quick and easy to learn and is valid for every kind of project. It’s not being applied consistently yet, despite being recommended by authorities such as the UK's Department for Transport and Treasury.

At Oxford Global Projects, we are committed to helping deliver projects on time, on budget and with the benefits promised. That’s why we are launching online RCF courses, open to all. If you are interested in learning more, please visit Oxford Global Projects Academy.

You may also be interested in

• Successful quality management requires expert risk management
• Making risk work for your project (🔒 APM Learning)
• How to make your project a success with Bent Flyvberg (APM Podcast)
• Being part of the APM community

_{[1] Flyvbjerg, Bent, Budzier, Alexander, 2021. The Oxford Global Projects Database.
[2] Flyvbjerg, Bent, 2017. Introduction: The Iron Law of Megaproject Management. The Oxford Handbook of Megaproject Management, Oxford University Press, Chapter 1, pp. 1-18.
[3] Kahneman, Daniel, 1994. New challenges to the rationality assumption. Journal of Institutional and Theoretical Economics, 150, pp 18-36
[4] Flyvbjerg, Bent, 2006. From Nobel Prize to Project Management: Getting Risks Right. Project Management Journal, vol. 37, no. 3, pp. 5-15.}