mlsanityData to 14 Oct 2026

The Tao of Agency: Autotelic AI, Embedded Agency and Dissolution of the Self

arXiv 2606.19924 ↗First posted 18 Jun 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Most artificial intelligence systems are built on the assumption that goals are exogenous and specified by the designer. Exploring what happens when an agent begins generating its own goals opens the field of autotelic AI. Agents are expected not merely to pursue objectives but to discover them. In this article, we trace its consequences through intrinsic motivation, resource-driven priors, causal-interventional learning, homeostasis, and embeddedness; the last of which is found to be a necessary but not sufficient condition for autotelic agency. Embeddedness individuates the agent at the cost of revealing that the individuation is non-unique, such that the same dynamics admit many valid partitions, each defining a different candidate self. The deepest problem with autotelic AI is therefore not how the agent generates goals, but how it generates and relativizes the self to which the goals are assigned. The agent must believe in its own boundary in order to act, and see through that boundary in order to understand. We consolidate these developments into a single framework and extend it along three directions: a quantum formulation in which the agent-environment cut becomes physical, a philosophical reading against non-dual contemplative traditions, and a concrete LLM-based agentic instantiation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 116 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

26 of 116 references.

References · 26

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2017Deep reinforcement learning from human preferencescited by 3,912
  5. 2023Let's Verify Step by Stepcited by 3,789
  6. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  7. 2016Concrete Problems in AI Safetycited by 1,642
  8. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  9. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  10. 2018Scalable agent alignment via reward modeling: a research directioncited by 447
  11. 2015Illuminating search spaces by mapping elitescited by 425
  12. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 356
14 more
  1. 2018Provably Efficient Maximum Entropy Explorationcited by 242
  2. 2020Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutionscited by 106
  3. 2020Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Surveycited by 85
  4. 2017Reinforcement Learning with a Corrupted Reward Channelcited by 70
  5. 2020Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Explorationcited by 68
  6. 2026Hyperagentscited by 56
  7. 2023Augmenting Autotelic Agents with Large Language Modelscited by 30
  8. 2019Embedded Agencycited by 19
  9. 2016When the map is better than the territorycited by 17
  10. 2019Categorizing Wireheading in Partially Embedded Agentscited by 5
  11. 2021QKSA: Quantum Knowledge Seeking Agentcited by 3
  12. 2025Quantum AIXI: Universal Intelligence via Quantum Informationcited by 2
  13. 2025Autotelic Reinforcement Learning: Exploring Intrinsic Motivations for Skill Acquisition in Open-Ended Environmentscited by 1
  14. 2025Formalizing Embeddedness Failures in Universal Artificial Intelligencecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Active Inference: A method for Phenotyping Agency in AI systems?
  2. 2026Critique of Agent Model
  3. 2025The Seeds of Scheming: Weakness of Will in the Building Blocks of Agentic Systems
  4. 2026Artificial Agency Program: Curiosity, compression, and communication in agents
  5. 2025Is the `Agent' Paradigm a Limiting Framework for Next-Generation Intelligent Systems?
  6. 2025Embedded Universal Predictive Intelligence: a coherent framework for multi-agent learning
  7. 2025Agency Is Frame-Dependent
  8. 2026A Mathematical Theory of Agency and Intelligence