# Modeling AGI Safety Frameworks with Causal Influence Diagrams

arXiv: https://arxiv.org/abs/1906.08663 · HTML: https://mlsanity.com/paper/1906.08663/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jun 2019.

## Built on by (most cited first, 8 of 8)

- [Alignment of Language Agents](https://mlsanity.com/paper/2103.14659/) (2021)
- [Agent Incentives: A Causal Perspective](https://mlsanity.com/paper/2102.01685/) (2021)
- [Counterfactual harm](https://mlsanity.com/paper/2204.12993/) (2022)
- [Worldwide AI Ethics: a review of 200 guidelines and recommendations for AI governance](https://mlsanity.com/paper/2206.11922/) (2022)
- [Equilibrium Refinements for Multi-Agent Influence Diagrams: Theory and Practice](https://mlsanity.com/paper/2102.05008/) (2021)
- [A Complete Criterion for Value of Information in Soluble Influence Diagrams](https://mlsanity.com/paper/2202.11629/) (2022)
- [Goal-Directedness is in the Eye of the Beholder](https://mlsanity.com/paper/2508.13247/) (2025)
- [Counterfactual Planning in AGI Systems](https://mlsanity.com/paper/2102.00834/) (2021)

## Abstract (arXiv)

Proposals for safe AGI systems are typically made at the level of frameworks, specifying how the components of the proposed system should be trained and interact with each other. In this paper, we model and compare the most promising AGI safety frameworks using causal influence diagrams. The diagrams show the optimization objective and causal assumptions of the framework. The unified representation permits easy comparison of frameworks and their assumptions. We hope that the diagrams will serve as an accessible and visual introduction to the main AGI safety frameworks.