Scaling Goal-based Exploration via Pruning Proto-goals
Not on the map. Why?
Abstract (arXiv)
One of the gnarliest challenges in reinforcement learning (RL) is exploration that scales to vast domains, where novelty-, or coverage-seeking behaviour falls short. Goal-directed, purposeful behaviours are able to overcome this, but rely on a good goal space. The core challenge in goal discovery is finding the right balance between generality (not hand-crafted) and tractability (useful, not too many). Our approach explicitly seeks the middle ground, enabling the human designer to specify a vast but meaningful proto-goal space, and an autonomous discovery process to refine this to a narrower space of controllable, reachable, novel, and relevant goals. The effectiveness of goal-conditioned exploration with the latter is then demonstrated in three challenging environments.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 40 of its 96 references that are in the map. First posted on arXiv 9 Feb 2023.
Builds on
40 of 96 references.
References · 40
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2016OpenAI Gymcited by 1,684
- 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
- 2017Hindsight Experience Replaycited by 1,096
- 2015Prioritized Experience Replaycited by 1,070
- 2018Exploration by Random Network Distillationcited by 966
- 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
- 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
- 2016Deep Exploration via Bootstrapped DQNcited by 518
- 2016The Option-Critic Architecturecited by 433
- 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
- 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
28 more
- 2018Large-Scale Study of Curiosity-Driven Learningcited by 277
- 2016Variational Intrinsic Controlcited by 262
- 2017Noisy Networks for Explorationcited by 262
- 2020First return, then explorecited by 256
- 2018Randomized Prior Functions for Deep Reinforcement Learningcited by 218
- 2020The NetHack Learning Environmentcited by 196
- 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 171
- 2017Parameter Space Noise for Explorationcited by 158
- 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
- 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 142
- 2020RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environmentscited by 135
- 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 120
- 2017A Laplacian Framework for Option Discovery in Reinforcement Learningcited by 108
- 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 100
- 2021MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Researchcited by 96
- 2020Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Surveycited by 83
- 2017Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learningcited by 79
- 2021The Option Keyboard: Combining Skills in Reinforcement Learningcited by 65
- 2021The Difficulty of Passive Learning in Deep Reinforcement Learningcited by 50
- 2019Discovery of Useful Questions as Auxiliary Taskscited by 45
- 2019Ray Interference: a Source of Plateaus in Deep Reinforcement Learningcited by 40
- 2021On Bonus-Based Exploration Methods in the Arcade Learning Environmentcited by 37
- 2021Discovery of Options via Meta-Learned Subgoalscited by 27
- 2022A Theory of Abstraction in Reinforcement Learningcited by 26
- 2021When should agents explore?cited by 20
- 2022The Alberta Plan for AI Researchcited by 17
- 2022Reward-Respecting Subtasks for Model-Based Reinforcement Learningcited by 15
- 2022World Value Functions: Knowledge Representation for Learning and Planningcited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.