mlsanityData to 7 Oct 2026

Safety Cases: How to Justify the Safety of Advanced AI Systems

arXiv 2403.10462 ↗First posted 15 Mar 2024cs.CY cs.AIcited by 48

Not on the map. Why?

Abstract (arXiv)

As AI systems become more advanced, companies and regulators will make difficult decisions about whether it is safe to train and deploy them. To prepare for these decisions, we investigate how developers could make a 'safety case,' which is a structured rationale that AI systems are unlikely to cause a catastrophe. We propose a framework for organizing a safety case and discuss four categories of arguments to justify safety: total inability to cause a catastrophe, sufficiently strong control measures, trustworthiness despite capability to cause harm, and -- if AI systems become much more powerful -- deference to credible AI advisors. We evaluate concrete examples of arguments in each category and outline how arguments could be combined to justify that AI systems are safe to deploy.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Mar 2024.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 24; 2026: 17.

48 papers.

Sort:

By area:

  1. 2025Reasoning Models Don't Always Say What They Thinkcited by 388
  2. 2024Frontier Models are Capable of In-context Schemingcited by 252
  3. 2025When Chain of Thought is Necessary, Language Models Struggle to Evade Monitorscited by 81
  4. 2024Sabotage Evaluations for Frontier Modelscited by 45
  5. 2025A sketch of an AI control safety casecited by 36
  6. 2025Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiescited by 36
  7. 2024Towards evaluations-based safety cases for AI schemingcited by 32
  8. 2024Games for AI Control: Models of Safety Evaluations of AI Deployment Protocolscited by 28
  9. 2026The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systemscited by 27
  10. 2025How to evaluate control measures for LLM agents? A trajectory from today to superintelligencecited by 20
  11. 2025LLMs Can Covertly Sandbag on Capability Evaluations Against Chain-of-Thought Monitoringcited by 17
  12. 2025An alignment safety case sketch based on debatecited by 12
36 more
  1. 2025Output Supervision Can Obfuscate the Chain of Thoughtcited by 11
  2. 2026Evaluating and Understanding Scheming Propensity in LLM Agentscited by 11
  3. 2024Societal Adaptation to Advanced AIcited by 10
  4. 2025The Singapore Consensus on Global AI Safety Research Prioritiescited by 9
  5. 2025STREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reportscited by 9
  6. 2025A Frontier AI Risk Management Framework: Bridging the Gap Between Current AI Practices and Established Risk Managementcited by 8
  7. 2025Control Tax: The Price of Keeping AI in Checkcited by 7
  8. 2025Beyond Black-Box Obfuscation: Mechanistic Analysis and Defense of White-Box Monitorscited by 6
  9. 2025Systematic Hazard Analysis for Frontier AI using STPAcited by 6
  10. 2025Practical challenges of control monitoring in frontier AI deploymentscited by 6
  11. 2026When Do LLM Preferences Predict Downstream Behavior?cited by 5
  12. 2024What AI evaluations for preventing catastrophic risks can and cannot docited by 4
  13. 2025An Example Safety Case for Safeguards Against Misusecited by 4
  14. 2025Misalignment or misuse? The AGI alignment tradeoffcited by 4
  15. 2025Manipulation Attacks by Misaligned AI: Risk Analysis and Safety Case Frameworkcited by 4
  16. 2025AI Companies Should Report Pre- and Post-Mitigation Safety Evaluationscited by 3
  17. 2024Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systemscited by 2
  18. 2025Justified Evidence Collection for Argument-based AI Fairness Assurancecited by 2
  19. 2025Corrigibility as a Singular Target: A Vision for Inherently Reliable Foundation Modelscited by 2
  20. 2025Against racing to AGI: Cooperation, deterrence, and catastrophic riskscited by 2
  21. 2025A Concrete Roadmap towards Safety Cases based on Chain-of-Thought Monitoringcited by 2
  22. 2026When Behavioral Safety Evaluation Fails: A Representation-Level Perspectivecited by 2
  23. 2026"Did you lie?" Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organismscited by 2
  24. 2025AI threats to national security can be countered through an incident regimecited by 1
  25. 2026Clear, Compelling Arguments: Rethinking the Foundations of Frontier AI Safety Casescited by 1
  26. 2026Institutional Red-Teaming: Deployment Rules, Not Just Models, Causally Shape Multi-Agent AI Safetycited by 1
  27. 2026Shared SFT Lessons Across Alignment, Model Organisms, and Toy Modelscited by 1
  28. 2026Questionnaire Responses Do not Capture the Safety of AI Agentscited by 0
  29. 2026Risk Reporting for Developers' Internal AI Model Usecited by 0
  30. 2026Training ML Models with Predictable Failurescited by 0
  31. 2026Alignment Plausibility: A New Standard for Assuring AI in Healthcarecited by 0
  32. 2026Intrinsic Structure: Spectral Identifiability for Mechanistic Interpretabilitycited by 0
  33. 2026Regime-Conditional Verification: Correctness Estimation for Adapting and Monitoring Safety Classifierscited by 0
  34. 2026Position: AI Is Not Ready for Strategic Conflictscited by 0
  35. 2026Monitor Jailbreaking: Evading Chain-of-Thought Monitoring Without Encoded Reasoningcited by 0
  36. 2026Comprehension Audits to Mitigate Risks from Automated AI Researchcited by 0