# Exploration by Distributional Reinforcement Learning

arXiv: https://arxiv.org/abs/1805.01907 · HTML: https://mlsanity.com/paper/1805.01907/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 May 2018.

## Built on by (most cited first, 13 of 13)

- [Exploration in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2205.00824/) (2022)
- [SENTINEL: Taming Uncertainty with Ensemble-based Distributional Reinforcement Learning](https://mlsanity.com/paper/2102.11075/) (2021)
- [Intrinsic Benefits of Categorical Distributional Loss: Uncertainty-aware Regularized Exploration in Reinforcement Learning](https://mlsanity.com/paper/2110.03155/) (2021)
- [Exploration via Epistemic Value Estimation](https://mlsanity.com/paper/2303.04012/) (2023)
- [Bayesian Distributional Policy Gradients](https://mlsanity.com/paper/2103.11265/) (2021)
- [Pitfall of Optimism: Distributional Reinforcement Learning by Randomizing Risk Criterion](https://mlsanity.com/paper/2310.16546/) (2023)
- [A unified uncertainty-aware exploration: Combining epistemic and aleatory uncertainty](https://mlsanity.com/paper/2401.02914/) (2024)
- [Optimal Transport-Guided Safety in Temporal Difference Reinforcement Learning](https://mlsanity.com/paper/2502.16328/) (2025)
- [Stochastic Path Planning in Correlated Obstacle Fields](https://mlsanity.com/paper/2509.19559/) (2025)
- [Exploration with Multi-Sample Target Values for Distributional Reinforcement Learning](https://mlsanity.com/paper/2202.02693/) (2022)
- [Bag of Policies for Distributional Deep Exploration](https://mlsanity.com/paper/2308.01759/) (2023)
- [DRL-ORA: Distributional Reinforcement Learning with Online Risk Adaption](https://mlsanity.com/paper/2310.05179/) (2023)
- [Distributional Active Inference](https://mlsanity.com/paper/2601.20985/) (2026)

## Abstract (arXiv)

We propose a framework based on distributional reinforcement learning and recent attempts to combine Bayesian parameter updates with deep reinforcement learning. We show that our proposed framework conceptually unifies multiple previous methods in exploration. We also derive a practical algorithm that achieves efficient exploration on challenging control tasks.