Shutdown Safety Valves for Advanced AI
Not on the map. Why?
Abstract (arXiv)
One common concern about advanced artificial intelligence is that it will prevent us from turning it off, as that would interfere with pursuing its goals. In this paper, we discuss an unorthodox proposal for addressing this concern: give the AI a (primary) goal of being turned off (see also papers by Martin et al., and by Goldstein and Robinson). We also discuss whether and under what conditions this would be a good idea.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 27 references that are in the map. First posted on arXiv 7 Mar 2026.
Builds on
18 of 27 references.
References · 18
- 2016Concrete Problems in AI Safetycited by 1,642
- 2018AI safety via debatecited by 400
- 2018Large-Scale Study of Curiosity-Driven Learningcited by 280
- 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 258
- 2019Optimal Policies Tend to Seek Powercited by 119
- 2016The Off-Switch Gamecited by 97
- 2023Scheming AIs: Will AIs fake alignment during training in order to get power?cited by 97
- 2017Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learningcited by 90
- 2025Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?cited by 75
- 2019Conservative Agency via Attainable Utility Preservationcited by 35
- 2017Should Robots be Obedient?cited by 20
- 2017Low Impact Artificial Intelligencescited by 11
6 more
- 2023Human Control: Definitions and Algorithmscited by 10
- 2016Death and Suicide in Universal Artificial Intelligencecited by 9
- 2017Incorrigibility in the CIRL Frameworkcited by 9
- 2017Guidelines for Artificial Intelligence Containmentcited by 7
- 2025AI Testing Should Account for Sophisticated Strategic Behaviourcited by 6
- 2025Off-Switching Not Guaranteedcited by 4
Built on by
Not cited yet.