mlsanityData to 7 Oct 2026

Explore-Go: Leveraging Exploration for Generalisation in Deep Reinforcement Learning

arXiv 2406.08069 ↗First posted 12 Jun 2024cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

One of the remaining challenges in reinforcement learning is to develop agents that can generalise to novel scenarios they might encounter once deployed. This challenge is often framed in a multi-task setting where agents train on a fixed set of tasks and have to generalise to new tasks. Recent work has shown that in this setting increased exploration during training can be leveraged to increase the generalisation performance of the agent. This makes sense when the states encountered during testing can actually be explored during training. In this paper, we provide intuition why exploration can also benefit generalisation to states that cannot be explicitly encountered during training. Additionally, we propose a novel method Explore-Go that exploits this intuition by increasing the number of states on which the agent trains. Explore-Go effectively increases the starting state distribution of the agent and as a result can be used in conjunction with most existing on-policy or off-policy reinforcement learning algorithms. We show empirically that our method can increase generalisation performance in an illustrative environment and on the Procgen benchmark.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 48 of its 63 references that are in the map. First posted on arXiv 12 Jun 2024.

Builds on

48 of 63 references.

References · 48

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  3. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  4. 2018Exploration by Random Network Distillationcited by 966
  5. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  6. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  7. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  8. 2021Domain Generalization with MixStylecited by 563
  9. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  10. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  11. 2018Quantifying Generalization in Reinforcement Learningcited by 345
  12. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 320
36 more
  1. 2020First return, then explorecited by 256
  2. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
  3. 2015Contextual Markov Decision Processescited by 213
  4. 2020Prioritized Level Replaycited by 190
  5. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  6. 2020RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environmentscited by 135
  7. 2020Phasic Policy Gradientcited by 121
  8. 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 111
  9. 2019Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learningcited by 102
  10. 2019Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneckcited by 101
  11. 2020Learning with AMIGo: Adversarially Motivated Intrinsic Goalscited by 86
  12. 2020On the Benefits of Invariance in Neural Networkscited by 78
  13. 2018Relational Deep Reinforcement Learningcited by 72
  14. 2017Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physicscited by 57
  15. 2022Exploration via Elliptical Episodic Bonusescited by 51
  16. 2021The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learningcited by 50
  17. 2020Neuroevolution of Self-Interpretable Agentscited by 48
  18. 2021Adversarially Guided Actor-Criticcited by 47
  19. 2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
  20. 2021Interesting Object, Curious Agent: Learning Task-Agnostic Explorationcited by 38
  21. 2021Unsupervised Visual Attention and Invariance for Reinforcement Learningcited by 30
  22. 2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 30
  23. 2021Robust Predictable Controlcited by 29
  24. 2023Planning Goals for Explorationcited by 28
  25. 2020Dynamics Generalization via Information Bottleneck in Deep Reinforcement Learningcited by 26
  26. 2022The good, the bad and the ugly sides of data augmentation: An implicit spectral regularization perspectivecited by 24
  27. 2023Explore to Generalize in Zero-Shot RLcited by 20
  28. 2023A Study of Global and Episodic Bonuses for Exploration in Contextual MDPscited by 19
  29. 2022Learning Instance-Specific Augmentations by Capturing Local Invariancescited by 15
  30. 2022LECO: Learnable Episodic Count for Task-Specific Intrinsic Rewardcited by 11
  31. 2022Exploring through Random Curiosity with General Value Functionscited by 10
  32. 2023Bad Habits: Policy Confounding and Out-of-Trajectory Generalization in RLcited by 10
  33. 2022Rethinking Value Function Learning for Generalization in Reinforcement Learningcited by 8
  34. 2021Don't Do What Doesn't Matter: Intrinsic Motivation with Action Usefulnesscited by 7
  35. 2021Explore and Control with Adversarial Surprisecited by 6
  36. 2023The Role of Diverse Replay for Generalisation in Reinforcement Learningcited by 1

Built on by

Not cited yet.