# Safety Cases: How to Justify the Safety of Advanced AI Systems

arXiv: https://arxiv.org/abs/2403.10462 · HTML: https://mlsanity.com/paper/2403.10462/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Mar 2024.

## Built on by (most cited first, 30 of 48)

- [Reasoning Models Don't Always Say What They Think](https://mlsanity.com/paper/2505.05410/) (2025)
- [Frontier Models are Capable of In-context Scheming](https://mlsanity.com/paper/2412.04984/) (2024)
- [When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors](https://mlsanity.com/paper/2507.05246/) (2025)
- [Sabotage Evaluations for Frontier Models](https://mlsanity.com/paper/2410.21514/) (2024)
- [A sketch of an AI control safety case](https://mlsanity.com/paper/2501.17315/) (2025)
- [Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities](https://mlsanity.com/paper/2502.05209/) (2025)
- [Towards evaluations-based safety cases for AI scheming](https://mlsanity.com/paper/2411.03336/) (2024)
- [Games for AI Control: Models of Safety Evaluations of AI Deployment Protocols](https://mlsanity.com/paper/2409.07985/) (2024)
- [The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systems](https://mlsanity.com/paper/2602.17753/) (2026)
- [How to evaluate control measures for LLM agents? A trajectory from today to superintelligence](https://mlsanity.com/paper/2504.05259/) (2025)
- [LLMs Can Covertly Sandbag on Capability Evaluations Against Chain-of-Thought Monitoring](https://mlsanity.com/paper/2508.00943/) (2025)
- [An alignment safety case sketch based on debate](https://mlsanity.com/paper/2505.03989/) (2025)
- [Evaluating and Understanding Scheming Propensity in LLM Agents](https://mlsanity.com/paper/2603.01608/) (2026)
- [Output Supervision Can Obfuscate the Chain of Thought](https://mlsanity.com/paper/2511.11584/) (2025)
- [Societal Adaptation to Advanced AI](https://mlsanity.com/paper/2405.10295/) (2024)
- [Control Tax: The Price of Keeping AI in Check](https://mlsanity.com/paper/2506.05296/) (2025)
- [The Singapore Consensus on Global AI Safety Research Priorities](https://mlsanity.com/paper/2506.20702/) (2025)
- [STREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reports](https://mlsanity.com/paper/2508.09853/) (2025)
- [A Frontier AI Risk Management Framework: Bridging the Gap Between Current AI Practices and Established Risk Management](https://mlsanity.com/paper/2502.06656/) (2025)
- [Practical challenges of control monitoring in frontier AI deployments](https://mlsanity.com/paper/2512.22154/) (2025)
- [When Do LLM Preferences Predict Downstream Behavior?](https://mlsanity.com/paper/2602.18971/) (2026)
- [Beyond Black-Box Obfuscation: Mechanistic Analysis and Defense of White-Box Monitors](https://mlsanity.com/paper/2505.14300/) (2025)
- [Systematic Hazard Analysis for Frontier AI using STPA](https://mlsanity.com/paper/2506.01782/) (2025)
- [Manipulation Attacks by Misaligned AI: Risk Analysis and Safety Case Framework](https://mlsanity.com/paper/2507.12872/) (2025)
- [What AI evaluations for preventing catastrophic risks can and cannot do](https://mlsanity.com/paper/2412.08653/) (2024)
- [An Example Safety Case for Safeguards Against Misuse](https://mlsanity.com/paper/2505.18003/) (2025)
- [Misalignment or misuse? The AGI alignment tradeoff](https://mlsanity.com/paper/2506.03755/) (2025)
- ["Did you lie?" Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organisms](https://mlsanity.com/paper/2606.12618/) (2026)
- [AI Companies Should Report Pre- and Post-Mitigation Safety Evaluations](https://mlsanity.com/paper/2503.17388/) (2025)
- [Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems](https://mlsanity.com/paper/2410.23472/) (2024)

## Abstract (arXiv)

As AI systems become more advanced, companies and regulators will make difficult decisions about whether it is safe to train and deploy them. To prepare for these decisions, we investigate how developers could make a 'safety case,' which is a structured rationale that AI systems are unlikely to cause a catastrophe. We propose a framework for organizing a safety case and discuss four categories of arguments to justify safety: total inability to cause a catastrophe, sufficiently strong control measures, trustworthiness despite capability to cause harm, and -- if AI systems become much more powerful -- deference to credible AI advisors. We evaluate concrete examples of arguments in each category and outline how arguments could be combined to justify that AI systems are safe to deploy.