mlsanityData to 14 Oct 2026

Enhancing Reinforcement Learning Agents with Local Guides

arXiv 2402.13930 ↗First posted 21 Feb 2024cs.LG cs.SY eess.SYcited by 4

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

This paper addresses the problem of integrating local guide policies into a Reinforcement Learning agent. For this, we show how to adapt existing algorithms to this setting before introducing a novel algorithm based on a noisy policy-switching procedure. This approach builds on a proper Approximate Policy Evaluation (APE) scheme to provide a perturbation that carefully leads the local guides towards better actions. We evaluated our method on a set of classical Reinforcement Learning problems, including safety-critical systems where the agent cannot enter some areas at the risk of triggering catastrophic consequences. In all the proposed environments, our agent proved to be efficient at leveraging those policies to improve the performance of any APE-based Reinforcement Learning algorithm, especially in its first learning stages.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 42 of its 92 references that are in the map. First posted on arXiv 21 Feb 2024.

Builds on

42 of 92 references.

References · 42

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  5. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  6. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  7. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  8. 2016Generative Adversarial Imitation Learningcited by 1,597
  9. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  10. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  11. 2017Constrained Policy Optimizationcited by 983
  12. 2018DeepMind Control Suitecited by 956
30 more
  1. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
  2. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  3. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 671
  4. 2019Behavior Regularized Offline Reinforcement Learningcited by 662
  5. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  6. 2018Is Q-learning Provably Efficient?cited by 487
  7. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  8. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 346
  9. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 340
  10. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 319
  11. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 298
  12. 2019A Theory of Regularized Markov Decision Processescited by 286
  13. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 238
  14. 2017Distral: Robust Multitask Reinforcement Learningcited by 233
  15. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 214
  16. 2022Jump-Start Reinforcement Learningcited by 112
  17. 2022Mildly Conservative Q-Learning for Offline Reinforcement Learningcited by 112
  18. 2018Playing hard exploration games by watching YouTubecited by 99
  19. 2020COG: Connecting New Skills to Past Experience with Offline Reinforcement Learningcited by 83
  20. 2018Kickstarting Deep Reinforcement Learningcited by 81
  21. 2020Safe Reinforcement Learning via Curriculum Inductioncited by 74
  22. 2020Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learningcited by 65
  23. 2019Wasserstein Adversarial Imitation Learningcited by 60
  24. 2022Reinforcement Learning with Sparse Rewards using Guidance from Offline Demonstrationcited by 54
  25. 2021Safe Reinforcement Learning Using Advantage-Based Interventioncited by 47
  26. 2018Observe and Look Further: Achieving Consistent Performance on Ataricited by 44
  27. 2020Towards Robust Data-Driven Control Synthesis for Nonlinear Systems with Actuation Uncertaintycited by 14
  28. 2019Imitation Learning from Pixel-Level Demonstrations by HashRewardcited by 6
  29. 2021Training Transition Policies via Distribution Matching for Complex Taskscited by 6
  30. 2020Combining Deep Reinforcement Learning And Local Control For The Acrobot Swing-up And Balance Taskcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 0; 2026: 2.

4 papers.

  1. 2024Contextualized Hybrid Ensemble Q-learning: Learning Fast with Control Priorscited by 5
  2. 2024Stepping Out of the Shadows: Reinforcement Learning in Shadow Modecited by 1
  3. 2026Expert Behavior Prior Reinforcement Learningcited by 0
  4. 2026Improving Online Reinforcement Learning via Bidirectional Behavior Prior Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A General Control-Theoretic Approach for Reinforcement Learning: Theory and Algorithms
  2. 2026Towards Optimal Policy Improvement
  3. 2024A Comprehensive Survey of Reinforcement Learning: From Algorithms to Practical Challenges
  4. 2023Understanding Reinforcement Learning Algorithms: The Progress from Basic Q-learning to Proximal Policy Optimization
  5. 2026Adaptive Expert Guidance for Efficient On-Policy Reinforcement Learning
  6. 2025A universal policy wrapper with guarantees
  7. 2024Efficient Exploration in Deep Reinforcement Learning: A Novel Bayesian Actor-Critic Algorithm
  8. 2024Enhancing Reinforcement Learning Through Guided Search