Data to 7 Oct 2026

Distributional Reinforcement Learning for Efficient Exploration

arXiv 1905.06125 ↗First posted 13 May 2019cs.LG cs.AI stat.MLcited by 59

Not on the map. Why?

Abstract (arXiv)

In distributional reinforcement learning (RL), the estimated distribution of value function models both the parametric and intrinsic uncertainties. We propose a novel and efficient exploration method for deep RL that has two components. The first is a decaying schedule to suppress the intrinsic uncertainty. The second is an exploration bonus calculated from the upper quantiles of the learned distribution. In Atari 2600 games, our method outperforms QR-DQN in 12 out of 14 hard games (achieving 483 % average gain across 49 games in cumulative rewards over QR-DQN with a big win in Venture). We also compared our algorithm with QR-DQN in a challenging 3D driving simulator (CARLA). Results show that our algorithm achieves near-optimal safety rewards twice faster than QRDQN.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 12; 2023: 16; 2024: 5; 2025: 8; 2026: 7.

59 papers.

Sort:

By area:

2022Pessimistic Bootstrapping for Uncertainty-Driven Offline Reinforcement Learningcited by 125
2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
2021Exponential Bellman Equation and Improved Regret Bounds for Risk-Sensitive Reinforcement Learningcited by 42
2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 30
2021DFAC Framework: Factorizing the Value Function via Quantile Mixture for Multi-Agent Distributional Q-Learningcited by 26
2023Distributional Soft Actor-Critic with Three Refinementscited by 22
2021On the Convergence and Optimality of Policy Gradient for Markov Coherent Riskcited by 19
2021Non-decreasing Quantile Function Network with Efficient Exploration for Distributional Reinforcement Learningcited by 15
2021GMAC: A Distributional Perspective on Actor-Critic Frameworkcited by 15
2022Conformal Off-policy Predictioncited by 14
2022The Sufficiency of Off-Policyness and Soft Clipping: PPO is still Insufficient according to an Off-Policy Measurecited by 12
2023Learning Risk-Aware Quadrupedal Locomotion using Distributional Reinforcement Learningcited by 12

38 more

2025Risk-Sensitive RL for Alleviating Exploration Dilemmas in Large Language Modelscited by 12
2025Value Flowscited by 12
2021Exploring the Training Robustness of Distributional Reinforcement Learning against Noisy State Observationscited by 11
2022Distributional Hamilton-Jacobi-Bellman Equations for Continuous-Time Reinforcement Learningcited by 9
2022Quantile Constrained Reinforcement Learning: A Reinforcement Learning Framework Constraining Outage Probabilitycited by 9
2022Distributional Reinforcement Learning with Regularized Wasserstein Losscited by 8
2023Diverse Projection Ensembles for Distributional Reinforcement Learningcited by 8
2021Damped Anderson Mixing for Deep Reinforcement Learning: Acceleration, Convergence, and Stabilizationcited by 7
2023Quasi-optimal Reinforcement Learning with Continuous Actionscited by 7
2023OVD-Explorer: Optimism Should Not Be the Sole Pursuit of Exploration in Noisy Environmentscited by 7
2024Pessimistic Value Iteration for Multi-Task Data Sharing in Offline Reinforcement Learningcited by 6
2022Adaptive Risk-Tendency: Nano Drone Navigation in Cluttered Environments with Distributional Reinforcement Learningcited by 5
2022Risk Perspective Exploration in Distributional Reinforcement Learningcited by 5
2023Wasserstein Actor-Critic: Directed Exploration via Optimism for Continuous-Actions Controlcited by 5
2024Model-Free Active Exploration in Reinforcement Learningcited by 5
2021Intrinsic Benefits of Categorical Distributional Loss: Uncertainty-aware Regularized Exploration in Reinforcement Learningcited by 4
2022How Does Return Distribution in Distributional Reinforcement Learning Help Optimization?cited by 4
2021Bayesian Distributional Policy Gradientscited by 3
2022Distributional Actor-Critic Ensemble for Uncertainty-Aware Continuous Controlcited by 3
2023Pitfall of Optimism: Distributional Reinforcement Learning by Randomizing Risk Criterioncited by 3
2023PACER: A Fully Push-forward-based Distributional Reinforcement Learning Algorithmcited by 2
2023Variance Control for Distributional Reinforcement Learningcited by 2
2024A unified uncertainty-aware exploration: Combining epistemic and aleatory uncertaintycited by 2
2026LAD: Learning Advantage Distribution for Reasoningcited by 2
2021ADER:Adapting between Exploration and Robustness for Actor-Critic Methodscited by 1
2022Mean-Semivariance Policy Optimization via Risk-Averse Reinforcement Learningcited by 1
2022Normality-Guided Distributional Reinforcement Learning for Continuous Controlcited by 1
2023Toward Risk-based Optimistic Exploration for Cooperative Multi-Agent Reinforcement Learningcited by 1
2023Distributional Reinforcement Learning with Dual Expectile-Quantile Regressioncited by 1
2023A Unified Framework for Factorizing Distributional Value Functions for Multi-Agent Reinforcement Learningcited by 1
2023Uncertainty-aware transfer across tasks using hybrid model-based successor feature reinforcement learningcited by 1
2024Enhanced Safety in Autonomous Driving: Integrating Latent State Diffusion Model for End-to-End Navigationcited by 1
2024State-Novelty Guided Action Persistence in Deep Reinforcement Learningcited by 1
2025Tackling Uncertainties in Multi-Agent Reinforcement Learning through Integration of Agent Termination Dynamicscited by 1
2025ADDQ: Adaptive Distributional Double Q-Learningcited by 1
2025Stochastic Path Planning in Correlated Obstacle Fieldscited by 1
2025LLM Agents Beyond Utility: An Open-Ended Perspectivecited by 1
2026Uncertainty-Aware Rank-One MIMO Q Network Framework for Accelerated Offline Reinforcement Learningcited by 1

50 most cited of 59 All 59 (JSON)