# An Example Safety Case for Safeguards Against Misuse

arXiv: https://arxiv.org/abs/2505.18003 · HTML: https://mlsanity.com/paper/2505.18003/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 23 May 2025.

## Builds on (most cited first, 13 of 13)

- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [Tree of Attacks: Jailbreaking Black-Box LLMs Automatically](https://mlsanity.com/paper/2312.02119/) (2023)
- [The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning](https://mlsanity.com/paper/2403.03218/) (2024)
- [Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming](https://mlsanity.com/paper/2501.18837/) (2025)
- [Safety Cases: How to Justify the Safety of Advanced AI Systems](https://mlsanity.com/paper/2403.10462/) (2024)
- [Structured access: an emerging paradigm for safe AI deployment](https://mlsanity.com/paper/2201.05159/) (2022)
- [A sketch of an AI control safety case](https://mlsanity.com/paper/2501.17315/) (2025)
- [Towards evaluations-based safety cases for AI scheming](https://mlsanity.com/paper/2411.03336/) (2024)
- [Does Unlearning Truly Unlearn? A Black Box Evaluation of LLM Unlearning Methods](https://mlsanity.com/paper/2411.12103/) (2024)
- [Safety case template for frontier AI: A cyber inability argument](https://mlsanity.com/paper/2411.08088/) (2024)
- [Safety cases for frontier AI](https://mlsanity.com/paper/2410.21572/) (2024)

## Built on by (most cited first, 4 of 4)

- [Dimensional Characterization and Pathway Modeling for Catastrophic AI Risks](https://mlsanity.com/paper/2508.06411/) (2025)
- [Clear, Compelling Arguments: Rethinking the Foundations of Frontier AI Safety Cases](https://mlsanity.com/paper/2603.08760/) (2026)
- [BELLS-O: Evaluating the Operational Trade-offs of LLM Supervision Systems](https://mlsanity.com/paper/2606.20668/) (2026)
- [The Safeguard Worked. Is the LLM System Safer?](https://mlsanity.com/paper/2609.00519/) (2026)

## Abstract (arXiv)

Existing evaluations of AI misuse safeguards provide a patchwork of evidence that is often difficult to connect to real-world decisions. To bridge this gap, we describe an end-to-end argument (a "safety case") that misuse safeguards reduce the risk posed by an AI assistant to low levels. We first describe how a hypothetical developer red teams safeguards, estimating the effort required to evade them. Then, the developer plugs this estimate into a quantitative "uplift model" to determine how much barriers introduced by safeguards dissuade misuse (https://www.aimisusemodel.com/). This procedure provides a continuous signal of risk during deployment that helps the developer rapidly respond to emerging threats. Finally, we describe how to tie these components together into a simple safety case. Our work provides one concrete path -- though not the only path -- to rigorously justifying AI misuse risks are low.