mlsanityData to 14 Oct 2026

Estimating Risk and Uncertainty in Deep Reinforcement Learning

arXiv 1905.09638 ↗First posted 23 May 2019cs.LG cs.AI stat.MLcited by 70

Not on the map. Why?

Cited by 20 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Reinforcement learning agents are faced with two types of uncertainty. Epistemic uncertainty stems from limited data and is useful for exploration, whereas aleatoric uncertainty arises from stochastic environments and must be accounted for in risk-sensitive applications. We highlight the challenges involved in simultaneously estimating both of them, and propose a framework for disentangling and estimating these uncertainties on learned Q-values. We derive unbiased estimators of these uncertainties and introduce an uncertainty-aware DQN algorithm, which we show exhibits safe learning behavior and outperforms other DQN variants on the MinAtar testbed.

Key facts. Cited by 70 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 13; 2022: 15; 2023: 15; 2024: 12; 2025: 7; 2026: 8.

70 papers.

Sort:

By area:

  1. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 302
  2. 2021Uncertainty Weighted Actor-Critic for Offline Reinforcement Learningcited by 126
  3. 2021Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulationcited by 85
  4. 2021Graph Posterior Network: Bayesian Predictive Uncertainty for Node Classificationcited by 80
  5. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 58
  6. 2023Anti-Exploration by Random Network Distillationcited by 50
  7. 2022A Review of Uncertainty for Deep Reinforcement Learningcited by 40
  8. 2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 31
  9. 2021Risk-Averse Bayes-Adaptive Reinforcement Learningcited by 29
  10. 2021How to Stay Curious while Avoiding Noisy TVs using Aleatoric Uncertainty Estimationcited by 27
  11. 2022Sample Efficient Deep Reinforcement Learning via Uncertainty Estimationcited by 27
  12. 2022Disentangling Epistemic and Aleatoric Uncertainty in Reinforcement Learningcited by 27
38 more
  1. 2023Distributional Soft Actor-Critic with Three Refinementscited by 25
  2. 2022One Risk to Rule Them All: A Risk-Sensitive Perspective on Model-Based Offline Reinforcement Learningcited by 23
  3. 2022General Intelligence Requires Rethinking Explorationcited by 20
  4. 2025GARDO: Reinforcing Diffusion Models without Reward Hackingcited by 17
  5. 2021SENTINEL: Taming Uncertainty with Ensemble-based Distributional Reinforcement Learningcited by 15
  6. 2021Safe Chance Constrained Reinforcement Learning for Batch Process Controlcited by 11
  7. 2024The Epistemic Uncertainty Hole: an issue of Bayesian Neural Networkscited by 11
  8. 2021A Hierarchical Variational Neural Uncertainty Model for Stochastic Video Predictioncited by 10
  9. 2023ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantagescited by 10
  10. 2023Decision-Making Under Uncertainty: Beyond Probabilitiescited by 9
  11. 2022Generating and Detecting True Ambiguity: A Forgotten Danger in DNN Supervision Testingcited by 8
  12. 2023Diverse Projection Ensembles for Distributional Reinforcement Learningcited by 8
  13. 2023OVD-Explorer: Optimism Should Not Be the Sole Pursuit of Exploration in Noisy Environmentscited by 7
  14. 2021Enabling risk-aware Reinforcement Learning for medical interventions through uncertainty decompositioncited by 6
  15. 2022Adaptive Risk-Tendency: Nano Drone Navigation in Cluttered Environments with Distributional Reinforcement Learningcited by 6
  16. 2022Optimistic Curiosity Exploration and Conservative Exploitation with Linear Reward Shapingcited by 6
  17. 2023Learning to Forecast Aleatoric and Epistemic Uncertainties over Long Horizon Trajectoriescited by 6
  18. 2023Uncertainty-driven Exploration Strategies for Online Grasp Learningcited by 6
  19. 2025Guaranteeing Out-Of-Distribution Detection in Deep RL via Transition Estimationcited by 6
  20. 2025Uncertainty Prioritized Experience Replaycited by 6
  21. 2024Model-Free Active Exploration in Reinforcement Learningcited by 5
  22. 2023Deep Gaussian Mixture Ensemblescited by 4
  23. 2024Multi-Fidelity Reinforcement Learning for Time-Optimal Quadrotor Re-planningcited by 4
  24. 2026MINT: Minimal Information Neuro-Symbolic Tree for Objective-Driven Knowledge-Gap Reasoning and Active Elicitationcited by 4
  25. 2021Learning to Be Cautiouscited by 3
  26. 2022Confident Neural Network Regression with Bootstrapped Deep Ensemblescited by 3
  27. 2022Distributional Actor-Critic Ensemble for Uncertainty-Aware Continuous Controlcited by 3
  28. 2022How to Enable Uncertainty Estimation in Proximal Policy Optimizationcited by 3
  29. 2023Mind the Uncertainty: Risk-Aware and Actively Exploring Model-Based Reinforcement Learningcited by 3
  30. 2023Pitfall of Optimism: Distributional Reinforcement Learning by Randomizing Risk Criterioncited by 3
  31. 2024Echoes of Socratic Doubt: Embracing Uncertainty in Calibrated Evidential Reinforcement Learningcited by 3
  32. 2025Bipedalism for Quadrupedal Robots: Versatile Loco-Manipulation through Risk-Adaptive Reinforcement Learningcited by 3
  33. 2026Neural Computerscited by 3
  34. 2023Distributional constrained reinforcement learning for supply chain optimizationcited by 2
  35. 2024A unified uncertainty-aware exploration: Combining epistemic and aleatory uncertaintycited by 2
  36. 2024Lifelong Reinforcement Learning via Neuromodulationcited by 2
  37. 2024Disentangling Uncertainty for Safe Social Navigation using Deep Reinforcement Learningcited by 2
  38. 2026RE-SAC: Disentangling aleatoric and epistemic risks in bus fleet control: A stable and robust ensemble DRL approachcited by 2

50 most cited of 70 All 70 (JSON)