# Off-Switching Not Guaranteed

arXiv: https://arxiv.org/abs/2502.08864 · HTML: https://mlsanity.com/paper/2502.08864/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 43 references that are in the map.
- First posted on arXiv 13 Feb 2025.

## Builds on (most cited first, 6 of 6)

- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [The Off-Switch Game](https://mlsanity.com/paper/1611.08219/) (2016)
- [Credal Bayesian Deep Learning](https://mlsanity.com/paper/2302.09656/) (2023)
- [Should Robots be Obedient?](https://mlsanity.com/paper/1705.09990/) (2017)
- [Incorrigibility in the CIRL Framework](https://mlsanity.com/paper/1709.06275/) (2017)
- [Rational Aversion to Information](https://mlsanity.com/paper/2309.12374/) (2023)

## Built on by (most cited first, 4 of 4)

- [Against Proxy Optimization](https://mlsanity.com/paper/2606.23597/) (2026)
- [AI Agents and Hard Choices](https://mlsanity.com/paper/2504.15304/) (2025)
- [Shutdown Safety Valves for Advanced AI](https://mlsanity.com/paper/2603.07315/) (2026)
- [Revisiting the shutdown problem](https://mlsanity.com/paper/2606.08296/) (2026)

## Abstract (arXiv)

Hadfield-Menell et al. (2017) propose the Off-Switch Game, a model of Human-AI cooperation in which AI agents always defer to humans because they are uncertain about our preferences. I explain two reasons why AI agents might not defer. First, AI agents might not value learning. Second, even if AI agents value learning, they might not be certain to learn our actual preferences.