mlsanityData to 14 Oct 2026

Distributional Actor-Critic Ensemble for Uncertainty-Aware Continuous Control

arXiv 2207.13730 ↗First posted 27 Jul 2022cs.LG cs.SY eess.SYcited by 3

Not on the map. Why?

Abstract (arXiv)

Uncertainty quantification is one of the central challenges for machine learning in real-world applications. In reinforcement learning, an agent confronts two kinds of uncertainty, called epistemic uncertainty and aleatoric uncertainty. Disentangling and evaluating these uncertainties simultaneously stands a chance of improving the agent's final performance, accelerating training, and facilitating quality assurance after deployment. In this work, we propose an uncertainty-aware reinforcement learning algorithm for continuous control tasks that extends the Deep Deterministic Policy Gradient algorithm (DDPG). It exploits epistemic uncertainty to accelerate exploration and aleatoric uncertainty to learn a risk-sensitive policy. We conduct numerical experiments showing that our variant of DDPG outperforms vanilla DDPG without uncertainty estimation in benchmark tasks on robotic control and power-grid optimization.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 48 of its 76 references that are in the map. First posted on arXiv 27 Jul 2022.

Builds on

48 of 76 references.

References · 48

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  5. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  6. 2015Prioritized Experience Replaycited by 1,099
  7. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
  8. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 819
  9. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  10. 2016Deep Exploration via Bootstrapped DQNcited by 527
  11. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 474
  12. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 369
36 more
  1. 2018Randomized Prior Functions for Deep Reinforcement Learningcited by 219
  2. 2018Distributed Distributional Deterministic Policy Gradientscited by 199
  3. 2020Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Criticscited by 165
  4. 2020SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learningcited by 152
  5. 2013Risk-sensitive Reinforcement Learningcited by 130
  6. 2014Algorithms for CVaR Optimization in MDPscited by 99
  7. 2018Uncertainty in Neural Networks: Approximately Bayesian Ensemblingcited by 94
  8. 2019Better Exploration with Optimistic Actor-Criticcited by 88
  9. 2019Fully Parameterized Quantile Function for Distributional Reinforcement Learningcited by 88
  10. 2020Distributional Soft Actor-Critic: Off-Policy Reinforcement Learning for Addressing Value Estimation Errorscited by 85
  11. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 73
  12. 2019Estimating Risk and Uncertainty in Deep Reinforcement Learningcited by 70
  13. 2021Risk-Averse Offline Reinforcement Learningcited by 65
  14. 2019Statistics and Samples in Distributional Reinforcement Learningcited by 63
  15. 2019Being Optimistic to Be Conservative: Quickly Learning a CVaR Policycited by 63
  16. 2019Distributional Reinforcement Learning for Efficient Explorationcited by 60
  17. 2018Information-Directed Exploration for Deep Reinforcement Learningcited by 52
  18. 2019Worst Cases Policy Gradientscited by 52
  19. 2020Mean-Variance Policy Iteration for Risk-Averse Reinforcement Learningcited by 28
  20. 2020Improving Robustness via Risk Averse Distributional Reinforcement Learningcited by 28
  21. 2021Deep Ensembles from a Bayesian Perspectivecited by 27
  22. 2015A Practical Guide to Robust Optimizationcited by 16
  23. 2021GMAC: A Distributional Perspective on Actor-Critic Frameworkcited by 16
  24. 2021SENTINEL: Taming Uncertainty with Ensemble-based Distributional Reinforcement Learningcited by 15
  25. 2021Risk-Conditioned Distributional Soft Actor-Critic for Risk-Sensitive Navigationcited by 15
  26. 2021Non-decreasing Quantile Function Network with Efficient Exploration for Distributional Reinforcement Learningcited by 15
  27. 2020Sample-based Distributional Policy Gradientcited by 14
  28. 2021PowerGym: A Reinforcement Learning Environment for Volt-Var Control in Power Distribution Systemscited by 14
  29. 2018ACE: An Actor Ensemble Algorithm for Continuous Control with Tree Searchcited by 12
  30. 2017Learning to Run with Actor-Critic Ensemblecited by 8
  31. 2021Continuous Control With Ensemble Deep Deterministic Policy Gradientscited by 8
  32. 2021Adaptively Calibrated Critic Estimates for Deep Reinforcement Learningcited by 6
  33. 2021Soft Actor-Critic With Integer Actionscited by 3
  34. 2018Distributional Advantage Actor-Criticcited by 2
  35. 2021Neural Network Ensembles: Theory, Training, and the Importance of Explicit Diversitycited by 2
  36. 2021ADER:Adapting between Exploration and Robustness for Actor-Critic Methodscited by 1

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspectivecited by 2
  2. 2025Ensemble Distribution Distillation for Self-Supervised Human Activity Recognitioncited by 0
  3. 2025UACER: An Uncertainty-Adaptive Critic Ensemble Framework for Robust Adversarial Reinforcement Learningcited by 0