# The Journey is the Reward: Unsupervised Learning of Influential Trajectories

arXiv: https://arxiv.org/abs/1905.09334 · HTML: https://mlsanity.com/paper/1905.09334/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2019.

## Built on by (most cited first, 2 of 2)

- [Experimental Evidence that Empowerment May Drive Exploration in Sparse-Reward Environments](https://mlsanity.com/paper/2107.07031/) (2021)
- [PreND: Enhancing Intrinsic Motivation in Reinforcement Learning through Pre-trained Network Distillation](https://mlsanity.com/paper/2410.01745/) (2024)

## Abstract (arXiv)

Unsupervised exploration and representation learning become increasingly important when learning in diverse and sparse environments. The information-theoretic principle of empowerment formalizes an unsupervised exploration objective through an agent trying to maximize its influence on the future states of its environment. Previous approaches carry certain limitations in that they either do not employ closed-loop feedback or do not have an internal state. As a consequence, a privileged final state is taken as an influence measure, rather than the full trajectory. We provide a model-free method which takes into account the whole trajectory while still offering the benefits of option-based approaches. We successfully apply our approach to settings with large action spaces, where discovery of meaningful action sequences is particularly difficult.