-By Caroline Metcalfe
Last August Proactima’s Henrik Bjelland and Caroline Metcalfe attended the European Safety and Reliability conference (ESREL) in Dublin, Ireland. Proactima’s contribution to the conference was titled “Outline of a risk management framework for future transportation systems” and was presented at the conference by Henrik. The article presented the work undertaken as part of the SIITS project to develop a risk management framework which can address the challenges faced by the adoption of novel technology into complex socio-technical systems. The framework aims to be adaptive such that as technology develops and changes the framework remains relevant and can adapt as systems change over time. The article was a collaboration between Proactima and University of Stavanger and was authored by Surbhi Bansal, Caroline Metcalfe, Roger Flage, Henrik Bjelland, Anders Jensen and Willy Røed.
ESREL is an annual conference and is attended by those in academia, industry and regulatory bodies and focuses on risk, reliability and safety. ESREL 2022 was co-hosted by Technological University Dublin (TU Dublin) and Trinity College, Dublin, who organized a great conference as well as an exciting overview of Irish culture, including Irish dancing at the conference dinner and a tour of the Old Library which houses the Book of Kells.
ESREL 2022 conference
ESREL 2022 took place from 28th August until 1st September 2022 and was attended by over 600 participants from around the globe. The conference was held on the Grangegorman Campus of TU Dublin. ESREL proceedings address many different aspects within safety, reliability and risk as well as their application in various disciplines such as health care, critical infrastructure and the chemical and process industry. To read more about the conference and access the book of proceedings see www.esrel2022.com.
Photo by Luke Tanis on Unsplash
There were many presentations that focused on the use of autonomous systems and how to assess the safety, reliability and risk of such systems. Several of these presentations focused on the use of autonomous systems within the transportation sector covering both how to assess these systems and how they are perceived and affect the different stakeholders involved within such complex socio-technical systems.
The development of better methods to assess complex socio-technical systems and the understanding of the ethical challenges is an important step in the introduction of autonomous technology into complex socio-technical systems. However, there is also a need for guidance in how to apply these methods and ensure the ethical challenges are addressed in the risk analysis and management of complex systems. The risk management framework developed as part of the SIITS project has an in-depth planning phase which includes elements to help understanding the various stakeholders involved, their values and perspectives of the risk problem and system being assessed and the use of this information to select the best methods for assessing the system.
International Workshops of Autonomous System Safety (IWASS)
Both Henrik and Caroline also participated in a pre-conference workshop hosted by IWASS (International Workshop on Autonomous Systems Safety). To read more about IWASS and read the proceedings of the workshop when published see https://www.risksciences.ucla.edu/iwass2022. The topic of the workshop for 2022 was “What do we mean be safe autonomous systems?”. Those involved had various backgrounds within academia, industry and regulators.
The topic of discussion “What do we mean by safe autonomous systems?” was broken down into several discussion points which were covered in the workshop, including:
- What do we mean by autonomous systems, how do they differ from automatic systems or technologies?
- What do we mean by a safe system and how to assess the safety of autonomous systems?
- What are the roles of standards and regulations in autonomous systems?
Some challenges discussed within the workshop were defining the “operational envelope” of an autonomous system, the transition from autonomous operation to human operator and how to define a safe autonomous system in relation to non-autonomous systems. These are important challenges that the developed framework aims to address. The inclusion of an initial description of the system guides the assessor to consider the different operational envelopes and environments of the system and how the different elements of the system work together to meet the functionality of the system. This should clarify the understanding of the system or show what needs to be understood better to be able to describe the system sufficiently. The top-down, iterative approach taken by the framework allows the assessor to understand what problem is to be addressed and at which level of detail, which was also covered in the discussion, especially in relation to the operational envelope of autonomous systems and the interaction between the autonomous system and human operator.
Presented risk management framework for future transportation systems
The framework presented by Henrik at ESREL 2022 has been developed to address the challenges that arise when new and emerging technology is adopted within socio-technical systems, resulting in the systems becoming more and more complex. The first step in developing the framework was to map the challenges of both current and future complex socio-technical systems. Relating these challenges to the current risk management frameworks, such as ISO 31000 and NS 5814 gave insight into which aspects of current frameworks need to be improved in order to assess future systems.
The process involved in the framework follows that of a plan, do, check, act structure which is common in existing risk management frameworks. The overall structure of the framework is that of an iterative process. It considers the complexity within the system and rather than trying to understand and model the entire system, it instead takes an incremental, top-down approach. This allows the system to first be understood from a high-level perspective, focusing on what is the function of the system rather than the individual components and how the different areas of the system interact to achieve the overall function of the system. Following the iterative process will build familiarity with the system and find the right level of detail required to assess and manage the risk for the different elements of the system. The different elements of the framework have been grouped into three phases; plan, assess and manage. The figure shows the structure and main elements of each of the three phases.
The process involved in the framework follows that of a plan, do, check, act structure which is common in existing risk management frameworks. The overall structure of the framework is that of an iterative process. It considers the complexity within the system and rather than trying to understand and model the entire system, it instead takes an incremental, top-down approach. This allows the system to first be understood from a high-level perspective, focusing on what is the function of the system rather than the individual components and how the different areas of the system interact to achieve the overall function of the system. Following the iterative process will build familiarity with the system and find the right level of detail required to assess and manage the risk for the different elements of the system. The different elements of the framework have been grouped into three phases; plan, assess and manage. The figure shows the structure and main elements of each of the three phases.
Risk management framework for future transportation systems
The plan phase aims to develop an understanding of the system context, frame the problem to be addressed and use this information to set the basis for the assess phase. Contextual understanding is developed by creating an initial system understanding of its function and interactions with other systems or actors, understanding the stakeholders involved and contemplate the different viewpoints and values involved. Profiling the risk and complexities of the system and problem to be addressed forms the basis for problem framing. This is then followed by gathering information on the relevant threats and hazards as well as exploring possible future scenarios. From this the evaluation criteria(s) for which the results of the assessment shall be evaluated can be decided. Finally, the risk management strategies and appropriate analysis tools and methods are considered before moving onto the assess phase.
The first step of the assess phase is to develop a detailed system description. This detailed description should be relevant to the perspective of the current problem framing and contextual understanding established in the plan phase. Using this system description, different scenarios which have been prioritized can be analyzed, and the potential consequences, associated uncertainties and strength of supporting knowledge are expressed. Finally, the risk can be evaluated against the criteria decided upon in the plan phase.
The final phase is the manage phase. Based on the evaluation of the risk in the assess phase, the risk treatment strategies can be determined. Performance monitoring indicators should be established to ensure that the chosen treatment strategy(s) have the intended effect and there are no inadvertent consequences. A clear plan for communicating the risk to the different stakeholders should be developed, where the best strategy(s) for each stakeholder group is decided. The final stage is to (re-)assess the risk management strategy(s) and determine if they are still relevant given the contextual understanding of the system and the risk problem. The process then returns to the plan phase where the knowledge gained in the latest iteration of the framework is used to improve the management of the risk within the system.
Next steps for the risk management framework
During his presentation, Henrik explained that the framework is a work in progress and thus changes are likely to be made as the SIITS project continues. The next step in the progress of the framework is to apply the framework in more pilot studies and see how practical the different aspects of the framework steps are and make the needed adjustments and revisions when required. Two such studies are currently underway. The first is focused on stakeholder analysis and value assessment as part assessing the risk in the development of the transportation system for an industrial area. The second is investigating the use of assurance cases in relation to risk assessment and management and exploring if this is something that should be included in the framework either as a choice of method or if the process of assurance cases can influence and guide changes to the process of the framework.
Many of the challenges related to autonomous systems are novel and thus the framework needs to be tested on relevant systems and revised to ensure that the final version can adapt and continue to be relevant for assessing and managing the risk related to the dynamic complex socio-technical systems we rely on for everyday life.
Photo by Christian Lue on Unsplash