# Learning telic-controllable state representations

arXiv: https://arxiv.org/abs/2406.14476 · HTML: https://mlsanity.com/paper/2406.14476/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 33 references that are in the map.
- First posted on arXiv 20 Jun 2024.

## Builds on (most cited first, 13 of 13)

- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Learning Invariant Representations for Reinforcement Learning without Reconstruction](https://mlsanity.com/paper/2006.10742/) (2020)
- [Automatic Goal Generation for Reinforcement Learning Agents](https://mlsanity.com/paper/1705.06366/) (2017)
- [Contrastive Learning as Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2206.07568/) (2022)
- [People construct simplified mental representations to plan](https://mlsanity.com/paper/2105.06948/) (2021)
- [Settling the Reward Hypothesis](https://mlsanity.com/paper/2212.10420/) (2022)
- [Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoning](https://mlsanity.com/paper/2111.03189/) (2021)
- [Deciding What to Learn: A Rate-Distortion Approach](https://mlsanity.com/paper/2101.06197/) (2021)
- [Exploring the hierarchical structure of human plans via program generation](https://mlsanity.com/paper/2311.18644/) (2023)
- [State Representation Learning for Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2205.01965/) (2022)
- [Building Minimal and Reusable Causal State Abstractions for Reinforcement Learning](https://mlsanity.com/paper/2401.12497/) (2024)
- [Goal-conditioned Offline Reinforcement Learning through State Space Partitioning](https://mlsanity.com/paper/2303.09367/) (2023)
- [States as goal-directed concepts: an epistemic approach to state-representation learning](https://mlsanity.com/paper/2312.02367/) (2023)

## Abstract (arXiv)

Computational models of purposeful behavior comprise both descriptive and prescriptive aspects, used respectively to ascertain and evaluate situations in the world. In reinforcement learning, prescriptive reward functions are assumed to depend on predefined and fixed descriptive state representations. Alternatively, these two aspects may emerge interdependently: goals can shape the acquired state representations and vice versa. Here, we present a computational framework for state representation learning in bounded agents, where descriptive and prescriptive aspects are coupled through the notion of goal-directed, or telic, states. We introduce the concept of telic-controllability to characterize the tradeoff between the granularity of a telic state representation and the policy complexity required to reach all telic states. We propose an algorithm for learning telic-controllable state representations, illustrating it using a simulated navigation task. Our framework highlights the role of deliberate ignorance -- knowing what to ignore -- for learning state representations that balance goal flexibility and cognitive complexity.