7. Analysis methods

So now that we have an idea of what criterion to use, how do we get there? What kind of analysis methods make best use of it?

There’s been a lot of interest lately in STAMP/STPA (Systems Theoretic Accident Model and Processes/Systems-Theoretic Process Analysis), developed by Nancy Leveson at MIT. And also the Bow Tie analysis method, which is intended to help identify safety threats, barriers that should prevent those threats from becoming hazards, and if a hazard occurs, barriers that should prevent the hazard from resulting in an accident.

The original Bow Tie method was intended to be the combination of a fault tree on one side and an event tree on the other. The fault tree described the potential failure modes of a system, and the event tree was used to map out how a hazard resulting from those failures could evolve to outcomes. Over time, it’s been modified into a purely qualitative barriers analysis on both sides. This is very well aligned with James Reason’s Swiss Cheese model of accident causation (more about that later).

Leveson has published an insightful critique of drawbacks of the Bow Tie method, one of which is that the method presumes that the analyst already knows all the potential threats and hazards. (Fault trees also have this issue). However, while STPA provides a much better analysis of where control structures exist and feedback could be needed, I think it also starts from step 2 rather than step 1.

So what is step 1? To me, it’s a more comprehensive exploration of how an initiating condition might develop into all plausible outcomes, which is best done with an event tree. The purpose of this is to map out what did happen, what should have happened, and what else could have happened instead. This event tree should be informed by human factors analysis so plausible alternative actions can be considered at every step. It would start with the initiating condition and map out all the plausible pathways, both safe and unsafe, based on known human tendencies and performance shaping factors. In my experience, this exercise has revealed previously unrecognized hazards and associated pathways to them. It can also help identify key points where the condition may be fragile to particularly dangerous but plausible decisions or actions. This can then point to where additional barriers (feedback, protections, etc.) may be needed.

Referring back to “Some models may be harmful”, I suggest that this type of model is intended to be exploratory rather than explanatory, to raise questions rather than to nail down decisions. Filling out as complete an event tree of plausible pathways based on human factors principles should, in my view, precede STPA, Bow Tie, or any other method that’s intended to help inform a safety decision. And to me, that's the proper role of a model: to make sure you've thought of everything. The problem with the finance industry leading up to the 2008 housing crisis was that they were letting the models think and decide for them.