Data to 7 Oct 2026

A Review of Uncertainty for Deep Reinforcement Learning

arXiv 2208.09052 ↗First posted 18 Aug 2022cs.LGcited by 36

Not on the map. Why?

Abstract (arXiv)

Uncertainty is ubiquitous in games, both in the agents playing games and often in the games themselves. Working with uncertainty is therefore an important component of successful deep reinforcement learning agents. While there has been substantial effort and progress in understanding and working with uncertainty for supervised learning, the body of literature for uncertainty aware deep reinforcement learning is less developed. While many of the same problems regarding uncertainty in neural networks for supervised learning remain for reinforcement learning, there are additional sources of uncertainty due to the nature of an interactable environment. In this work, we provide an overview motivating and presenting existing techniques in uncertainty aware deep reinforcement learning. These works show empirical benefits on a variety of reinforcement learning tasks. This work serves to help to centralize the disparate results and promote future research in this area.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 35 of its 39 references that are in the map. First posted on arXiv 18 Aug 2022.

Builds on

35 of 39 references.

References · 35

2017Proximal Policy Optimization Algorithmscited by 14,441
2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
2015Continuous control with deep reinforcement learningcited by 3,279
2013Playing Atari with Deep Reinforcement Learningcited by 3,211
2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
2018Exploration by Random Network Distillationcited by 966
2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 903
2021A Minimalist Approach to Offline Reinforcement Learningcited by 882

23 more

2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
2021A Survey of Uncertainty in Deep Neural Networkscited by 675
2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
2020MOPO: Model-based Offline Policy Optimizationcited by 630
2016Deep Exploration via Bootstrapped DQNcited by 518
2017Distributional Reinforcement Learning with Quantile Regressioncited by 464
2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 307
2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 297
2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
2018Randomized Prior Functions for Deep Reinforcement Learningcited by 218
2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 211
2017Deep Exploration via Randomized Value Functionscited by 155
2021Uncertainty Weighted Actor-Critic for Offline Reinforcement Learningcited by 125
2022Pessimistic Bootstrapping for Uncertainty-Driven Offline Reinforcement Learningcited by 125
2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 108
2018Uncertainty in Neural Networks: Approximately Bayesian Ensemblingcited by 94
2022Why So Pessimistic? Estimating Uncertainties for Offline RL through Ensembles, and Why Their Independence Matterscited by 74
2019Estimating Risk and Uncertainty in Deep Reinforcement Learningcited by 68
2021Ensemble Bootstrapping for Q-Learningcited by 30
2021How to Stay Curious while Avoiding Noisy TVs using Aleatoric Uncertainty Estimationcited by 27
2022Sample Efficient Deep Reinforcement Learning via Uncertainty Estimationcited by 26
2017Efficient exploration with Double Uncertain Value Networkscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 8; 2024: 9; 2025: 8; 2026: 10.

36 papers.

Sort:

By area:

2023Uncertainty-Penalized Reinforcement Learning from Human Feedback with Diverse Reward LoRA Ensemblescited by 55
2025General agents contain world modelscited by 14
2023A Simple Unified Uncertainty-Guided Framework for Offline-to-Online Reinforcement Learningcited by 8
2023Estimation and Inference in Distributional Reinforcement Learningcited by 8
2022Epistemic Monte Carlo Tree Searchcited by 7
2026ALOE: Action-Level Off-Policy Evaluation for Vision-Language-Action Model Post-Trainingcited by 7
2023On the Theory of Risk-Aware Agents: Bridging Actor-Critic and Economicscited by 6
2026Failure Identification in Imitation Learning Via Statistical and Semantic Filteringcited by 6
2024Advancing Investment Frontiers: Industry-grade Deep Reinforcement Learning for Portfolio Optimizationcited by 4
2026MINT: Minimal Information Neuro-Symbolic Tree for Objective-Driven Knowledge-Gap Reasoning and Active Elicitationcited by 4
2024Echoes of Socratic Doubt: Embracing Uncertainty in Calibrated Evidential Reinforcement Learningcited by 3
2024A unified uncertainty-aware exploration: Combining epistemic and aleatory uncertaintycited by 2

24 more

2024Disentangling Uncertainty for Safe Social Navigation using Deep Reinforcement Learningcited by 2
2023Meta Generative Flow Networks with Personalization for Task-Specific Adaptationcited by 1
2023Robustness to Multi-Modal Environment Uncertainty in MARL using Curriculum Learningcited by 1
2023Uncertainty-aware transfer across tasks using hybrid model-based successor feature reinforcement learningcited by 1
2023Modeling Risk in Reinforcement Learning: A Literature Mappingcited by 1
2024Temporal-Aware Deep Reinforcement Learning for Energy Storage Bidding in Energy and Contingency Reserve Marketscited by 1
2024vMFER: Von Mises-Fisher Experience Resampling Based on Uncertainty of Gradient Directions for Policy Improvementcited by 1
2024Toward Debugging Deep Reinforcement Learning Programs with RLExplorercited by 1
2025Quantum-Inspired Reinforcement Learning in the Presence of Epistemic Ambivalencecited by 1
2025Evaluating Uncertainty in Deep Gaussian Processescited by 1
2025GenAI-based Multi-Agent Reinforcement Learning towards Distributed Agent Intelligence: A Generative-RL Agent Perspectivecited by 1
2026A Blueprint for Equilibrium-Based Differentiable Continuous-Variable Thermodynamic Computingcited by 1
2024Games of Knightian Uncertainty as AGI testbedscited by 0
2024Generalized Gaussian Temporal Difference Error for Uncertainty-aware Reinforcement Learningcited by 0
2025YRC-Bench: A Benchmark for Learning to Coordinate with Expertscited by 0
2025Post-Hoc Split-Point Self-Consistency Verification for Efficient, Unified Quantification of Aleatoric and Epistemic Uncertainty in Deep Learningcited by 0
2025Uncertainty-Based Smooth Policy Regularisation for Reinforcement Learning with Few Demonstrationscited by 0
2025Deep Gaussian Process Proximal Policy Optimizationcited by 0
2026Selecting Offline Reinforcement Learning Algorithms for Stochastic Network Controlcited by 0
2026Oracle-Guided Soft Shielding for Safe Move Prediction in Chesscited by 0
2026Uncertainty-Aware and Temporally Regulated Expert Advice in Reinforcement Learning for Autonomous Drivingcited by 0
2026COP-Q: Safety-First Reinforcement Learning for Robot Control via Cholesky-Ordered Projectioncited by 0
2026Uncertainty-aware reinforcement learning for chemical language modelscited by 0
2026Uncertainty-Driven Replay Memory for Reinforcement Learningcited by 0