mlsanityData to 14 Oct 2026

Human Control: Definitions and Algorithms

arXiv 2305.19861 ↗First posted 31 May 2023cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

How can humans stay in control of advanced artificial intelligence systems? One proposal is corrigibility, which requires the agent to follow the instructions of a human overseer, without inappropriately influencing them. In this paper, we formally define a variant of corrigibility called shutdown instructability, and show that it implies appropriate shutdown behavior, retention of human autonomy, and avoidance of user harm. We also analyse the related concepts of non-obstruction and shutdown alignment, three previously proposed algorithms for human control, and one new algorithm.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 20 of its 42 references that are in the map. First posted on arXiv 31 May 2023.

Builds on

20 of 42 references.

References · 20

  1. 2018Scalable agent alignment via reward modeling: a research directioncited by 447
  2. 2018AI safety via debatecited by 400
  3. 2016Cooperative Inverse Reinforcement Learningcited by 300
  4. 2019Optimal Policies Tend to Seek Powercited by 119
  5. 2017AI Safety Gridworldscited by 110
  6. 2019Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspectivecited by 108
  7. 2016The Off-Switch Gamecited by 97
  8. 2016Research Priorities for Robust and Beneficial Artificial Intelligencecited by 74
  9. 2021Agent Incentives: A Causal Perspectivecited by 51
  10. 2019Conservative Agency via Attainable Utility Preservationcited by 35
  11. 2018Towards Formal Definitions of Blameworthiness, Intention, and Moral Responsibilitycited by 32
  12. 2020Avoiding Side Effects By Considering Future Taskscited by 32
8 more
  1. 2022Counterfactual harmcited by 29
  2. 2017Should Robots be Obedient?cited by 20
  3. 2022Path-Specific Objectives for Safer Agent Incentivescited by 20
  4. 2016Death and Suicide in Universal Artificial Intelligencecited by 9
  5. 2017Incorrigibility in the CIRL Frameworkcited by 9
  6. 2019Corrigibility with Utility Preservationcited by 8
  7. 2021Definitions of intent suitable for algorithmscited by 8
  8. 2017'Indifference' methods for managing agent rewardscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 2; 2026: 2.

10 papers.

  1. 2024Open-Endedness is Essential for Artificial Superhuman Intelligencecited by 80
  2. 2024Visibility into AI Agentscited by 79
  3. 2024Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementationcited by 27
  4. 2024The Shutdown Problem: An AI Engineering Puzzle for Decision Theoristscited by 22
  5. 2025Emergent Risk Awareness in Rational Agents under Resource Constraintscited by 5
  6. 2024Quantifying stability of non-power-seeking in artificial agentscited by 2
  7. 2024Towards Shutdownable Agents via Stochastic Choicecited by 2
  8. 2026Golden Handcuffs make safer AI agentscited by 1
  9. 2025Password-Activated Shutdown Protocols for Misaligned Frontier Agentscited by 0
  10. 2026Shutdown Safety Valves for Advanced AIcited by 0