# Shutdown Safety Valves for Advanced AI

arXiv: https://arxiv.org/abs/2603.07315 · HTML: https://mlsanity.com/paper/2603.07315/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 27 references that are in the map.
- First posted on arXiv 7 Mar 2026.

## Builds on (most cited first, 18 of 18)

- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [AI safety via debate](https://mlsanity.com/paper/1805.00899/) (2018)
- [Large-Scale Study of Curiosity-Driven Learning](https://mlsanity.com/paper/1808.04355/) (2018)
- [Risks from Learned Optimization in Advanced Machine Learning Systems](https://mlsanity.com/paper/1906.01820/) (2019)
- [Optimal Policies Tend to Seek Power](https://mlsanity.com/paper/1912.01683/) (2019)
- [The Off-Switch Game](https://mlsanity.com/paper/1611.08219/) (2016)
- [Scheming AIs: Will AIs fake alignment during training in order to get power?](https://mlsanity.com/paper/2311.08379/) (2023)
- [Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learning](https://mlsanity.com/paper/1711.06782/) (2017)
- [Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?](https://mlsanity.com/paper/2502.15657/) (2025)
- [Conservative Agency via Attainable Utility Preservation](https://mlsanity.com/paper/1902.09725/) (2019)
- [Should Robots be Obedient?](https://mlsanity.com/paper/1705.09990/) (2017)
- [Low Impact Artificial Intelligences](https://mlsanity.com/paper/1705.10720/) (2017)
- [Human Control: Definitions and Algorithms](https://mlsanity.com/paper/2305.19861/) (2023)
- [Death and Suicide in Universal Artificial Intelligence](https://mlsanity.com/paper/1606.00652/) (2016)
- [Incorrigibility in the CIRL Framework](https://mlsanity.com/paper/1709.06275/) (2017)
- [Guidelines for Artificial Intelligence Containment](https://mlsanity.com/paper/1707.08476/) (2017)
- [AI Testing Should Account for Sophisticated Strategic Behaviour](https://mlsanity.com/paper/2508.14927/) (2025)
- [Off-Switching Not Guaranteed](https://mlsanity.com/paper/2502.08864/) (2025)

## Abstract (arXiv)

One common concern about advanced artificial intelligence is that it will prevent us from turning it off, as that would interfere with pursuing its goals. In this paper, we discuss an unorthodox proposal for addressing this concern: give the AI a (primary) goal of being turned off (see also papers by Martin et al., and by Goldstein and Robinson). We also discuss whether and under what conditions this would be a good idea.