7. Summary and conclusion
We began this paper presenting the logical sequence upon which our reasoning is based: Validation of a risk analysis should be based on how well the risk analysis supports risk management, and assessing how well the risk analysis supports risk management should include considering how well it supports the actual, real-world decision making process, and the roles trust, third party review and stakeholder review and acceptance may play in that. Then we reworded that logic: Validation for risk management must be considered from a systems perspective, and presented that system in Fig. 1. That figure, in turn, provided the structure for a series of eight functions of risk analysis, and within those functions sixteen tests of validity, each test worded as a question. Each test is paired with a discussion of the shortfalls associated with failing that test. That Fig. 1 is divided into three domains: Analysts’ Domain, Users’ Domain and Analysis Community Domain. The Analysts’ Domain is the one domain implicitly assumed in any discussion of risk analysis validation we have seen. We add to that perspective the fact that there is a Users’ Domain separating any risk analysis from the final consequences and residual risk, which is the only thing that actually matters. That Users’ Domain includes the risk management decision process, desired implementation of risk management actions, and the actual risk management actions achieved based on the acceptance or denial of those actions, where that acceptance/denial is a result of implementers and stakeholder acceptance, which in turn is a result of the risk analysis report and its transparency and documentation. Underlying both the Analysts’ and Users’ Domains is the Analysis Community Domain, which could provide a ‘‘Culture of Analysis Quality” where the sixteen validation tests listed here would be enforced by both risk analysts and commissioners of risk analyses. Though we end by admitting that it is quite challenging to identify how to create that culture.
In conclusion, we have here presented a very broad, insightful systems perspective on the matter of how to establish risk analysis validity, including sixteen concrete tests of validity.
The authors are grateful to the four anonymous reviewers for their useful comments and suggestions to the original version of this paper. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.