Guided Exploration in Reinforcement Learning via Monte Carlo Critic Optimization
Not on the map. Why?
Abstract (arXiv)
The class of deep deterministic off-policy algorithms is effectively applied to solve challenging continuous control problems. Current approaches commonly utilize random noise as an exploration method, which has several drawbacks, including the need for manual adjustment for a given task and the absence of exploratory calibration during the training process. We address these challenges by proposing a novel guided exploration method that uses an ensemble of Monte Carlo Critics for calculating exploratory action correction. The proposed method enhances the traditional exploration scheme by dynamically adjusting exploration. Subsequently, we present a novel algorithm that leverages the proposed exploratory module for both policy and critic modification. The presented algorithm demonstrates superior performance compared to modern reinforcement learning algorithms across a variety of problems in the DMControl suite.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 25 of its 48 references that are in the map. First posted on arXiv 25 Jun 2022.
Builds on
25 of 48 references.
References · 25
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2015Continuous control with deep reinforcement learningcited by 3,407
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
- 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
- 2016OpenAI Gymcited by 1,736
- 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
- 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
- 2018Exploration by Random Network Distillationcited by 993
- 2018DeepMind Control Suitecited by 956
- 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
13 more
- 2017Deep Reinforcement Learning that Matterscited by 754
- 2017Noisy Networks for Explorationcited by 266
- 2019Self-Supervised Exploration via Disagreementcited by 265
- 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
- 2018Visual Reinforcement Learning with Imagined Goalscited by 251
- 2020Never Give Up: Learning Directed Exploration Strategiescited by 212
- 2020Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Criticscited by 165
- 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 141
- 2021Discovering and Achieving Goals via World Modelscited by 131
- 2019Better Exploration with Optimistic Actor-Criticcited by 88
- 2021A Survey of Exploration Methods in Reinforcement Learningcited by 73
- 2019Leveraging exploration in off-policy algorithms via normalizing flowscited by 46
- 2021Solving Continuous Control with Episodic Memorycited by 8
Built on by
Citing papers per year: 2024: 1.
1 paper.