Systematic Hazard Analysis for Frontier AI using STPA
Not on the map. Why?
Abstract (arXiv)
All of the frontier AI companies have published safety frameworks where they define capability thresholds and risk mitigations that determine how they will safely develop and deploy their models. Adoption of systematic approaches to risk modelling, based on established practices used in safety-critical industries, has been recommended, however frontier AI companies currently do not describe in detail any structured approach to identifying and analysing hazards. STPA (Systems-Theoretic Process Analysis) is a systematic methodology for identifying how complex systems can become unsafe, leading to hazards. It achieves this by mapping out controllers and controlled processes then analysing their interactions and feedback loops to understand how harmful outcomes could occur (Leveson & Thomas, 2018). We evaluate STPA's ability to broaden the scope, improve traceability and strengthen the robustness of safety assurance for frontier AI systems. Applying STPA to the threat model and scenario described in 'A Sketch of an AI Control Safety Case' (Korbak et al., 2025), we derive a list of Unsafe Control Actions. From these we select a subset and explore the Loss Scenarios that lead to them if left unmitigated. We find that STPA is able to identify causal factors that may be missed by unstructured hazard analysis methodologies thereby improving robustness. We suggest STPA could increase the safety assurance of frontier AI when used to complement or check coverage of existing AI governance techniques including capability thresholds, model evaluations and emergency procedures. The application of a systematic methodology supports scalability by increasing the proportion of the analysis that could be conducted by LLMs, reducing the burden on human domain experts.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 9 of its 47 references that are in the map. First posted on arXiv 2 Jun 2025.
Builds on
9 of 47 references.
References · 9
- 2023AI Control: Improving Safety Despite Intentional Subversioncited by 229
- 2024Safety Cases: How to Justify the Safety of Advanced AI Systemscited by 48
- 2025A sketch of an AI control safety casecited by 39
- 2023Risk assessment at AGI companies: A review of popular risk assessment techniques from other safety-critical industriescited by 19
- 2024Safety case template for frontier AI: A cyber inability argumentcited by 19
- 2025A Frontier AI Risk Management Framework: Bridging the Gap Between Current AI Practices and Established Risk Managementcited by 8
- 2025Safety Cases: A Scalable Approach to Frontier AI Safetycited by 8
- 2024From Silos to Systems: Process-Oriented Hazard Analysis for AI Systemscited by 6
- 2025Emerging Practices in Frontier AI Safety Frameworkscited by 6
Built on by
Citing papers per year: 2025: 2; 2026: 4.
6 papers.
- 2026Towards Verifiably Safe Tool Use for LLM Agentscited by 36
- 2025Dimensional Characterization and Pathway Modeling for Catastrophic AI Riskscited by 1
- 2026Compiling Sufficient Governance Context from Declared Losses and Reachable States: Exact Observation-Contract Synthesis with Cardinality and Cost Objectivescited by 1
- 2025HySafe-AI: Hybrid Safety Architectural Analysis Framework for AI Systems: A Case Studycited by 0
- 2026Quantifying System-Level Harms from AI Adoption in Complex Sociotechnical Systemscited by 0
- 2026AI Persuasion as a Threat to Human Controlcited by 0
- 2026Compiling Sufficient Governance Context from Declared Losses and Reachable States: Exact Observation-Contract Synthesis with Cardinality and Cost Objectives22 Sep 2026
- 2026AI Persuasion as a Threat to Human Control13 Sep 2026
- 2026Quantifying System-Level Harms from AI Adoption in Complex Sociotechnical Systems24 Aug 2026
- 2026Towards Verifiably Safe Tool Use for LLM Agents12 Jan 2026
- 2025Dimensional Characterization and Pathway Modeling for Catastrophic AI Risks8 Aug 2025
- 2025HySafe-AI: Hybrid Safety Architectural Analysis Framework for AI Systems: A Case Study23 Jul 2025