# The Tao of Agency: Autotelic AI, Embedded Agency and Dissolution of the Self

arXiv: https://arxiv.org/abs/2606.19924 · HTML: https://mlsanity.com/paper/2606.19924/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 116 references that are in the map.
- First posted on arXiv 18 Jun 2026.

## Builds on (most cited first, 26 of 26)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Unifying Count-Based Exploration and Intrinsic Motivation](https://mlsanity.com/paper/1606.01868/) (2016)
- [Scalable agent alignment via reward modeling: a research direction](https://mlsanity.com/paper/1811.07871/) (2018)
- [Illuminating search spaces by mapping elites](https://mlsanity.com/paper/1504.04909/) (2015)
- [Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents](https://mlsanity.com/paper/2505.22954/) (2025)
- [Provably Efficient Maximum Entropy Exploration](https://mlsanity.com/paper/1812.02690/) (2018)
- [Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions](https://mlsanity.com/paper/2003.08536/) (2020)
- [Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Survey](https://mlsanity.com/paper/2012.09830/) (2020)
- [Reinforcement Learning with a Corrupted Reward Channel](https://mlsanity.com/paper/1705.08417/) (2017)
- [Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Exploration](https://mlsanity.com/paper/2002.09253/) (2020)
- [Hyperagents](https://mlsanity.com/paper/2603.19461/) (2026)
- [Augmenting Autotelic Agents with Large Language Models](https://mlsanity.com/paper/2305.12487/) (2023)
- [Embedded Agency](https://mlsanity.com/paper/1902.09469/) (2019)
- [When the map is better than the territory](https://mlsanity.com/paper/1612.09592/) (2016)
- [Categorizing Wireheading in Partially Embedded Agents](https://mlsanity.com/paper/1906.09136/) (2019)
- [QKSA: Quantum Knowledge Seeking Agent](https://mlsanity.com/paper/2107.01429/) (2021)
- [Quantum AIXI: Universal Intelligence via Quantum Information](https://mlsanity.com/paper/2505.21170/) (2025)
- [Autotelic Reinforcement Learning: Exploring Intrinsic Motivations for Skill Acquisition in Open-Ended Environments](https://mlsanity.com/paper/2502.04418/) (2025)
- [Formalizing Embeddedness Failures in Universal Artificial Intelligence](https://mlsanity.com/paper/2505.17882/) (2025)

## Abstract (arXiv)

Most artificial intelligence systems are built on the assumption that goals are exogenous and specified by the designer. Exploring what happens when an agent begins generating its own goals opens the field of autotelic AI. Agents are expected not merely to pursue objectives but to discover them. In this article, we trace its consequences through intrinsic motivation, resource-driven priors, causal-interventional learning, homeostasis, and embeddedness; the last of which is found to be a necessary but not sufficient condition for autotelic agency. Embeddedness individuates the agent at the cost of revealing that the individuation is non-unique, such that the same dynamics admit many valid partitions, each defining a different candidate self. The deepest problem with autotelic AI is therefore not how the agent generates goals, but how it generates and relativizes the self to which the goals are assigned. The agent must believe in its own boundary in order to act, and see through that boundary in order to understand. We consolidate these developments into a single framework and extend it along three directions: a quantum formulation in which the agent-environment cut becomes physical, a philosophical reading against non-dual contemplative traditions, and a concrete LLM-based agentic instantiation.