Posted on

The Future of Risk Assessment?

Risk assessment is used to hugely-beneficial effect in many industries, ranging from off-shore oil and gas to nuclear and transportation. It is also used to plan for health epidemics, food safety and flooding. Generally the methods deployed are highly sophisticated, science-based and provide useful information for decision makers.

In recent years risk assessment has also been applied to public space and public activities, often using, by necessity, much simpler protocols such as HSE’s ‘Five steps to risk assessment’ (http://www.hse.gov.uk/pubns/indg163.pdf), or devices known as risk matrices of the type shown below which may range from 2×2 to even 10×10 cells.

These matrices, and the way they are used, are a source of worry for some eminent risk practitioners. For one thing they are usually qualitative in that there are no numerical values on either axis and both risk and consequence are subjectively-defined. What, for instance, does it mean to say something has a ‘medium’ likelihood? And in the case of public space is it the likelihood of something happening to one individual member of the public on a single visit, or on multiple visits over a year, or is it the likelihood that any member of the public might come to grief during, say, the next month? There are numerous possibilities but most matrices seem to leave this question unanswered despite having massive implications for their meaning.

Another technical anxiety is that users quite often number the cells along each axis from, say, one to three in the case of the matrix shown, and then multiply them together to generate a score for each cell. In this way the top right cell in the matrix shown would score 9, and the bottom left 1. This is troubling for several reasons. For one thing the axes are qualitative and labelling them from 1 to 3 produces ordinal numbers only (as in first, second, third etc), and not cardinal numbers. It never has made sense to multiply ordinal numbers together.

Another matter is that when numbers are multiplied, however ill-advisedly, they are sometimes said to be used as a means of prioritising and the aim of many risk assessors appears to be to shift hazards falling in the red-coloured boxes to the green ones. The requirement under the law, however, and which is deeply rational, is that control measures should be implemented if they are reasonably practicable. This is a much more sophisticated concept and is not replaceable by fiddling with coloured squares on grids. It could mean that hazards in higher-scoring boxes should not be subject to further control measures because there are none which are reasonably practicable.

An altogether different issue comes up when considering the risk assessment of public space and public activities, and this relates to the reason for their provision which is, obviously, their benefits. Public facilities, such as city squares, tree-lined streets, ponds and fountains, riverside walks, parks, playgrounds, forests, church yards, sites of cultural heritage, national parks, village fêtes, carnivals and sports of whatever kind produce benefits of all kinds ranging from health (physical, emotional and social) to intellectual stimulation, beauty and tranquillity. The problem is that these benefits get no mention in standard risk assessment protocols. It is as if the decision about whether, say, a public activity should be permitted or curtailed could be made purely by thinking about its inherent dangers and without any need to consider the benefits of the activity. This would be an irrational decision process. There is almost always some sort of trade-off between control measures and benefits when dealing with public places and public activities and it is essential that this be recognised, not ignored.

In effect, what has to be done is to weigh the benefits of, say, an unspoilt riverside walk and its risks, and make a decision on whether it should be left as it is or modified. This approach, of weighing risks and benefits, is what is increasingly being referred to as risk-benefit assessment (RBA). If you try to make this decision without thinking about either one of these commodities, the risks or the benefits, I for one could not conceive what was going on in the risk assessor’s head. But I suspect these for me unfathomable thought processes are quite common given that standard risk assessment protocols only refer to one side of this equation, leaving it open to ignore or forget the other.

Collectively these issues suggest that risk assessment as used in public life may require radical surgery.

Posted on

The Manchester Hole Case

This involved the tragic death by drowning of a young school boy while on an adventure holiday in Yorkshire. The school party was exploring an underground cave in Nidderdale known as Manchester Hole when the cave experienced an highly unusual flooding event. The case was brought by the Health and Safety Executive who sought to prosecute North Yorkshire County Council who ran the outdoor centre. I was involved as one of several expert witnesses. The case was in court for six weeks, with ultimately the HSE losing, much to its chagrin since it seemed to challenge some of its fundamental beliefs.

A fuller account of the case can be found in Horizons volume 51 published by the Outdoor Learning Institute in December 2010 (http://www.outdoor-learning.org/Default.aspx?tabid=137), but the main issues raised in the case from my perspective as a risk professional were:

  • Should young people be deliberately exposed to risk?

HSE’s published position on this was decidedly ambiguous. According to its 1999 publication ‘Adventure activities centres: five steps to risk assessment’:

“Adventure activities aim to allow young people to develop by meeting challenges they do not necessarily face every day and to experience a sense of achievement in overcoming them. Some degree of risk is unavoidable if the sense of adventure and excitement is to be achieved. However, it is important to remember that adventure activities should only create a sense of adventure and excitement and not cause harm.”

This statement presents a field day for policy analysts, for on the one hand it implies that young people need to meet real challenges and in so doing some risk is inevitable, but then informs that adventure should be fake (only sensed) and  not cause harm (HSE’s emphasis). Apart from its obvious inconsistencies, this contrasts sharply with HSE’s previous support of the Play Safety Forum’s policy statement which says quite simply that “Children need and want to take risks…”.

  • Should risk be negligible?

The Prosecution argued that an acceptable level of risk of serious injury or death is that it should be “negligible.” Since the HSWA requires an employer to ‘ensure’ the health and safety of employees etc, so, prima facie, if an accident occurs s/he has failed. The burden then falls upon the employer to prove that they had done everything which was reasonably practicable to ensure health and safety. The HSE alleged that because an accident had happened this proved that all reasonable measures had not been taken and NYCC had to prove otherwise, which is a curiously circuitous argument for an agency which purports to have a risk-based philosophy.

Another issue of considerable interest was whether the concept of reasonable practicability should encompass consideration of the benefits of such activities. In a civil case it likely would, but this was a criminal case where the answer was less clear cut. However, Justice Wilkie, in directing the jury, explicitly identified as a material consideration ‘the benefits of conducting the activity,’ just as Lord Reid had done in a much earlier civil case.

  • Should adventure activities be managed like factories?

Oral evidence by the HSE in court was in part along the lines that health and safety systems should be designed, implemented and managed in a local authority environment in exactly the same way as in any other organisation, and frequent reference was made to one of HSE’s best-selling texts, known as ‘Successful health and safety management’ or ‘HSG65’, described on HSE’s website as an “over-arching guide on the essential philosophy of good health and safety.” HSG65 is, however, primarily written for industry, not public life, though it could be concluded from this case that HSE sees no difference. But there are differences. Local authorities are providing and managing public services for their public benefits. HSG65 was written in a benefits vacuum where such things are barely acknowledged if at all. This might conceivably be alright in an industrial environment – you wouldn’t expect factory employees to have to walk along a plank to get to the canteen, or office workers to abseil to the toilet, but in public life pitting yourself against these challenges can have benefits.

  • Have you got the right management system?

Another aspect of the HSE’s case against NYCC was targeted on alleged deficiencies of NYCC’s management system. To this end HSG65 was also frequently cited. The fact is, though, that HSG65 is an advisory document, the advice given is based on some unknown person’s opinion, the evidence-base that it is cost-effective is unidentified, and there is no legal requirement to abide by that opinion. It was not at all obvious that the kind of management system described therein was suitable for an education authority or a county council.

  • What and who make adventure activities as safe as they are?

Much time was allocated in court to arguments about the technicalities of risk assessment and who should be involved in performing this required task.

On the technical side, the prosecution’s witnesses described the essential intricacies and interlinking of generic, site-specific, and dynamic risk assessment, which, it was said, comprised a ‘three-legged stool.’ The intended inference was that all three legs had to be equally robust to avoid catastrophic failure, but there was also an impression created that this stool was the rock upon which the safety of adventure activities rested. From a personal point of view, and as I said in court, if my children were going on an adventure activity, I should not want to rely upon such a device. Far more important would be the skill, knowledge and competence of the team leaders!

With respect to who should be doing the risk assessments, HSE’s position was that professional health and safety people employed in NYCC’s headquarters should have had a greater role in compiling the adventure activity risk assessments, whereas defence witnesses, from around the country, poured varying levels of scorn upon this proposition.

The research on expertise, with which I have become familiar in recent years, has led me to believe that there are different kinds of expertise, some of which are appropriate for certain tasks, but not for others. At the bottom of the heap is ‘beer mat knowledge’ – the sort of wisdom you gain from reading about bird flu on the back of a beer mat! In sharp contrast, interactional expertise is what you get from, say, years of participation in canoeing. The implication here is that unless there is strong evidence that anyone on NYCC’s central health and safety team had maximal ‘interactional expertise’ as in potholing experience – through years actually potholing themselves or through years and years actually discoursing with cavers about technical matters – then the only deep expertise about the dangers and so forth would have lain with the adventure leaders themselves. This position accorded with the views of the adventure leaders who gave evidence in court.

Posted on

Risk-benefit assessment

The move to get risk-benefit assessment recognised as a suitable and sufficient form of risk assessment is gathering momentum but still has hurdles to overcome. It seems that some agencies do not want to concede that the benefits of public space and activities should be a primary consideration in determining how safety from injury decisions should be made. This may be because this would result in a transfer of power because, to make balancing decisions, you would need to know about both the risks and the benefits of some place or activity and traditional H&S exponents may know little or nothing about benefits.

The case for RBA was strongly put in 2008 by Play England and the government in its ‘Managing risk in play provision – implementation guide’ (http://www.playengland.org.uk/resources/managing-risk-in-play-provision-implementation-guide.aspx). The case has more recently been expressed by the National Tree Safety Group in its ‘Common sense risk management of trees’ published by the Forestry Commission (http://www.forestry.gov.uk/pdf/FCMS024.pdf/$FILE/FCMS024.pdf), as well as by the long-standing Visitor Safety in the Countryside Group (http://vscg.co.uk/). Other organisations with strong interests include the Association of Heads of Outdoor Education Centres and the English Outdoor Council.

Overall, there is a widespread desire to get the benefits of public life back on the agenda of health and safety. Paradoxically, one of these benefits is health. But as the Trades Union Congress has put it, “Sadly we have a society that seems to see preventing injury as being more important than preventing illness.”

Posted on

Some legal questions

Three questions come to mind right now. These are:

a) According to the most widely used criminal law definition of reasonable practicability a responsible party should always err on the side of safety according to the principle known as ‘gross disproportion’. The question is whether or not this continues to be logical either in general, or in the context of public activities specifically.

b) Whether the requirements set by the law in terms of safety should be seen as a minimum requirement.

c) Whether the law is supportive of the use of risk-benefit assessment.

In response to a) the Löfstedt report provides a much-to-be-welcomed blast of intellect and fresh air because of its emphasis upon a return to risk-based decision making (as opposed to hazard identification and remediation), the use of scientific evidence, and proportionality. Nonetheless, there remain serious hurdles to overcome and his continuing commitment to taking this forward is to be welcomed.

One hurdle is the oft-touted formula “the disproportion must always be gross” which implies a requirement to err on the side of safety to a degree which is “gross.” This position is given added emphasis by HSE in its ‘Principles and guidelines’ document to assist its own officers in which it says “but the disproportion must always be gross,” itself an apparent reference to situations in which risks may already be small. How, though, is it possible to square gross disproportion in low risk situations with Löfstedt’s proportionality?

The origin of the “the disproportion must always be gross” formula is Lord Asquith’s summing up of the 1949 Edwards v National Coal Board case.  However, there are reasons why this might now be challenged both in general and in the particular case of public space. The ‘in general’ argument is to do with the way in which safety has been valued by society. Back in the 1940s the traditional method used by government agencies was simply to do with the value of the lost earnings of an hypothetical deceased person. This came up with rather niggardly valuations of safety and it may well be that the courts were aware of this in their deliberations, so adding a weighting factor under the banner of ‘gross disproportion.’ Come the 1980s, however, the method of arriving at these valuations was switched by the Department of Transport (others followed their lead) to what is known as ‘Willingness-to-pay” (WTP). WTP methods give much higher valuations, currently in the region of £1.5 million per life saved, being based on what consumers are prepared to pay for reducing their own risks. It can be argued that applying a ‘gross disproportion’ weighting to consumers’ own valuations would be tantamount to giving consumers something they have not asked for and do not want!

As for the specific context of public space, the argument has a second thread. This is that if it is accepted that decisions about what to allow in public space require a balancing judgement to be made about the benefits and the risks of that space, then to subsequently apply gross disproportion would be equivalent to loading one scale pan in favour of the other. Why?

In response to b), it is sometimes said and written that the requirements of the law with respect to safety should be seen as a minimum requirement. Prima facie, this is a morally sustainable argument and hard to challenge. But how valid is it?

The legal requirement under the HSWA is to implement safety measures which are reasonably practicable. This implies consideration of the effectiveness of safety measures in reducing risk versus the cost, time and difficulty of applying those measures. If the risk reduction benefit outweighs the latter, the measure must be implemented, and if not, it is not necessary to do so, though safety advocates might still want to go ahead and implement. However, from the position of a neutral bystander, say the proverbial ‘man on the Clapham omnibus,’ going beyond the point of rational decision making as implied by reasonable practicability is equivalent to saying that the views of the man on that bus should be ignored.

The fundamental issue is, of course, that if an agency providing some public good decides it wants to gold-plate its safety measures, then the related costs are going to have to be picked up somewhere and that will ultimately mean the taxpayer. So the quest for higher levels of safety than those reasonably conceived is itself morally questionable. Those resources spent on one thing could have been spent on something else which is possibly valued more highly.

As for c), the answer is that Civil Law has for a long time recognised four factors in determining whether a duty holder has done all that is reasonable in taking care. The factors are the prior level of risk, the severity of the possible consequence, the practicability of control measures and the social utility of the activity. The latter is of special interest in the context of public space and activities because social utility is why they exist. So in Civil Law the use of some kind of intellectual weighing of the benefits of space versus the risks would seem to be an accepted part of the process. However, in criminal law, under the HSWA, it is less clear. Lord Asquith’s 1949 definition of reasonable practicability identified the first three of the above factors but not the fourth. But maybe he didn’t need to because the case he was thinking about was the death of a miner in a coal mine where social utility was not an issue. So perhaps the position is best summarised as one of uncertainty. The courts have not had sufficient criminal cases to work on in which this dimension was relevant (but see The Manchester Hole case).