Problems with probability


Dr. David Hillson

Risk is defined in two dimensions: uncertainty and effect on objectives. It is common to use the terms 'probability' and 'impact' to describe these two dimensions, and assessing the significance of any given risk means considering both. It is relatively simple to assess effect on objectives, since this merely requires defining the situation after the risk has occurred, and then imagining what happens:

dice“If this risk occurred, what would the effect be?” Probability is not so easy however. Risk practitioners and project teams alike experience repeated difficulty in assessing the probability that a given risk might occur. There are a number of reasons for this.

  • Language. In English, different words are often used interchangeably to describe the uncertainty dimension of a risk, such as 'probability', 'frequency', 'likelihood' or 'chance'. In fact these do not mean the same thing, and confusion can arise if the terms are misused. For example “frequency” describes how often an event or set of circumstances is expected to occur based on previous experience, either in a period of time (e.g. once per year) or in a number of trials (e.g. seven times out of ten). So frequency really applies to repeatable events. This is not the same as “probability” which is a statistical term describing how likely a single uncertain event or set of circumstances is
    to occur. One solution is to use a more general term such as 'likelihood', and recognise two variants called 'probability' (for single events) and 'frequency' (for repeatable events).
  • Format. The uncertainty dimension of a risk can be expressed in several ways, including both numerical and textual formats, such as: 35%, 'once per month', 2:7, 'unlikely', 'one in six times', 10-4, 'low probability', 0.2, and so on. Most people have problems interpreting different numerical formats, and even the textual phrases can mean different things. This problem can best be overcome by education, as well as using a set of agreed definitions which everyone understands.
  • Subjectivity. Assessment of probability requires forming an opinion about a future event or set of circumstances which have not yet happened. Different people will take different views about the future, and there is no “single right answer” since the future has not yet happened. Risk probability cannot be measured, only estimated. Assessments of the uncertain future are influenced by many factors, including perceptual filters, motivational bias, cognitive bias, or
    subconscious heuristics. The solution here is to take a team-based approach, exploring different perspectives, examining underlying assumptions, and reaching consensus wherever possible. Sources of bias should also be understood and corrected where possible.
  • Lack of data. Some risks have never been experienced before, especially those relating to the unique aspects of projects. In other cases, even though a risk might have been encountered previously, there may be no record of its existence due to absence of a learning mechanism (such as a knowledge base or checklist). As a result there is no body of evidence to assist in estimating the probability of occurrence of these novel risks. Addressing these shortfalls requires acknowledging that some areas lack relevant previous experience, as well as implementing an effective lessons-to-be-learned process (e.g. a post-project review).

All this matters for two reasons:

  • Faulty probability assessment means risks will be wrongly prioritised, leading to a failure to focus on the most significant risks, selection of inappropriate responses, inability to manage risks effectively, and loss of confidence in the risk process.
  • Sound assessment of risk probability improves the understanding of each risk, allowing appropriate prioritisation, better response selection, enhanced risk management effectiveness, and more reliable achievement of project and business objectives.

We need to understand the problems associated with assessing probability, and take action to address the concerns, by using appropriate language and formats, identifying and managing sources of bias, learning lessons to improve the effectiveness of the probability assessment process, and monitoring risk management performance to determine the accuracy of assessed risk probability.




Please consider allowing cookies to be able to share this page on social media sites.

Change cookie settings
No history has been recorded.
Back to top