mlsanityData to 7 Oct 2026

Will artificial agents pursue power by default?

arXiv 2506.06352 ↗First posted 2 Jun 2025cs.AI cs.CYcited by 3

Not on the map. Why?

Abstract (arXiv)

Researchers worried about catastrophic risks from advanced AI have argued that we should expect sufficiently capable AI agents to pursue power over humanity because power is a convergent instrumental goal, something that is useful for a wide range of final goals. Others have recently expressed skepticism of these claims. This paper aims to formalize the concepts of instrumental convergence and power-seeking in an abstract, decision-theoretic framework, and to assess the claim that power is a convergent instrumental goal. I conclude that this claim contains at least an element of truth, but might turn out to have limited predictive utility, since an agent's options cannot always be ranked in terms of power in the absence of substantive information about the agent's final goals. However, the fact of instrumental convergence is more predictive for agents who have a good shot at attaining absolute or near-absolute power.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 10 of its 32 references that are in the map. First posted on arXiv 2 Jun 2025.

Builds on

10 of 32 references.

References · 10

  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  2. 2021Convex optimizationcited by 2,479
  3. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  4. 2020Artificial Intelligence, Values and Alignmentcited by 443
  5. 2022The Alignment Problem from a Deep Learning Perspectivecited by 246
  6. 2023An Overview of Catastrophic AI Riskscited by 214
  7. 2019Optimal Policies Tend to Seek Powercited by 115
  8. 2022Is Power-Seeking AI an Existential Risk?cited by 101
  9. 2023Power-seeking can be probable and predictive for trained agentscited by 23
  10. 2022Parametrically Retargetable Decision-Makers Tend To Seek Powercited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.