# The Potential of the Return Distribution for Exploration in RL

arXiv: https://arxiv.org/abs/1806.04242 · HTML: https://mlsanity.com/paper/1806.04242/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2018.

## Built on by (most cited first, 3 of 3)

- [Pitfall of Optimism: Distributional Reinforcement Learning by Randomizing Risk Criterion](https://mlsanity.com/paper/2310.16546/) (2023)
- [ADDQ: Adaptive Distributional Double Q-Learning](https://mlsanity.com/paper/2506.19478/) (2025)
- [Learning to Play: Reinforcement Learning and Games](https://mlsanity.com/paper/2610.17391/) (2026)

## Abstract (arXiv)

This paper studies the potential of the return distribution for exploration in deterministic reinforcement learning (RL) environments. We study network losses and propagation mechanisms for Gaussian, Categorical and Gaussian mixture distributions. Combined with exploration policies that leverage this return distribution, we solve, for example, a randomized Chain task of length 100, which has not been reported before when learning with neural networks.