# Learning Purposeful Behaviour in the Absence of Rewards

arXiv: https://arxiv.org/abs/1605.07700 · HTML: https://mlsanity.com/paper/1605.07700/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2016.

## Built on by (most cited first, 6 of 6)

- [Temporal Abstraction in Reinforcement Learning with the Successor Representation](https://mlsanity.com/paper/2110.05740/) (2021)
- [Deep Laplacian-based Options for Temporally-Extended Exploration](https://mlsanity.com/paper/2301.11181/) (2023)
- [Discovery of Options via Meta-Learned Subgoals](https://mlsanity.com/paper/2102.06741/) (2021)
- [Post-Training Large Language Models via Reinforcement Learning from Self-Feedback](https://mlsanity.com/paper/2507.21931/) (2025)
- [Model-Based Exploration in Monitored Markov Decision Processes](https://mlsanity.com/paper/2502.16772/) (2025)
- [Hierarchical Reinforcement Learning in Complex 3D Environments](https://mlsanity.com/paper/2302.14451/) (2023)

## Abstract (arXiv)

Artificial intelligence is commonly defined as the ability to achieve goals in the world. In the reinforcement learning framework, goals are encoded as reward functions that guide agent behaviour, and the sum of observed rewards provide a notion of progress. However, some domains have no such reward signal, or have a reward signal so sparse as to appear absent. Without reward feedback, agent behaviour is typically random, often dithering aimlessly and lacking intentionality. In this paper we present an algorithm capable of learning purposeful behaviour in the absence of rewards. The algorithm proceeds by constructing temporally extended actions (options), through the identification of purposes that are "just out of reach" of the agent's current behaviour. These purposes establish intrinsic goals for the agent to learn, ultimately resulting in a suite of behaviours that encourage the agent to visit different parts of the state space. Moreover, the approach is particularly suited for settings where rewards are very sparse, and such behaviours can help in the exploration of the environment until reward is observed.