mlsanityData to 14 Oct 2026

Cell-Free Latent Go-Explore

arXiv 2208.14928 ↗First posted 31 Aug 2022cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce Latent Go-Explore (LGE), a simple and general approach based on the Go-Explore paradigm for exploration in reinforcement learning (RL). Go-Explore was initially introduced with a strong domain knowledge constraint for partitioning the state space into cells. However, in most real-world scenarios, drawing domain knowledge from raw observations is complex and tedious. If the cell partitioning is not informative enough, Go-Explore can completely fail to explore the environment. We argue that the Go-Explore approach can be generalized to any environment without domain knowledge and without cells by exploiting a learned latent representation. Thus, we show that LGE can be flexibly combined with any strategy for learning a latent representation. Our results indicate that LGE, although simpler than Go-Explore, is more robust and outperforms state-of-the-art algorithms in terms of pure exploration on multiple hard-exploration environments including Montezuma's Revenge. The LGE implementation is available as open-source at https://github.com/qgallouedec/lge.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 43 of its 63 references that are in the map. First posted on arXiv 31 Aug 2022.

Builds on

43 of 63 references.

References · 43

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2017Neural Discrete Representation Learningcited by 4,712
  4. 2015Continuous control with deep reinforcement learningcited by 3,407
  5. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  6. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  7. 2017Hindsight Experience Replaycited by 1,136
  8. 2018Exploration by Random Network Distillationcited by 993
  9. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  10. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  11. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  12. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 474
31 more
  1. 2020Planning to Explore via Self-Supervised World Modelscited by 368
  2. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 269
  3. 2020First return, then explorecited by 266
  4. 2019Self-Supervised Exploration via Disagreementcited by 265
  5. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  6. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 212
  7. 2020The NetHack Learning Environmentcited by 198
  8. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 194
  9. 2020Reward-Free Exploration for Reinforcement Learningcited by 177
  10. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 174
  11. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 174
  12. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 166
  13. 2018State Representation Learning for Control: An Overviewcited by 127
  14. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 119
  15. 2018Count-Based Exploration with the Successor Representationcited by 112
  16. 2021APS: Active Pretraining with Successor Featurescited by 112
  17. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 102
  18. 2020Fast active learning for pure exploration in reinforcement learningcited by 92
  19. 2020Automatic Curriculum Learning through Value Disagreementcited by 87
  20. 2020Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Surveycited by 85
  21. 2017Surprise-Based Intrinsic Motivation for Deep Reinforcement Learningcited by 83
  22. 2018CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learningcited by 79
  23. 2020Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Explorationcited by 68
  24. 2021panda-gym: Open-source goal-conditioned environments for robotic learningcited by 59
  25. 2019Making Efficient Use of Demonstrations to Solve Hard Exploration Problemscited by 45
  26. 2021On Bonus-Based Exploration Methods in the Arcade Learning Environmentcited by 37
  27. 2019Robust Recovery Controller for a Quadrupedal Robot using Deep Reinforcement Learningcited by 32
  28. 2020Novelty Search in Representational Space for Sample Efficient Explorationcited by 26
  29. 2021Understanding the World Through Actioncited by 17
  30. 2020PBCS : Efficient Exploration and Exploitation Using a Synergy between Reinforcement Learning and Motion Planningcited by 12
  31. 2020Hindsight Expectation Maximization for Goal-conditioned Reinforcement Learningcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025Analytical Survey of Learning with Low-Resource Data: From Analysis to Investigationcited by 5
  2. 2026Decoupling Exploration and Policy Optimization: Uncertainty Guided Tree Search for Hard Explorationcited by 1
  3. 2025Meta-Learning to Explore via Memory Density Feedbackcited by 0
  4. 2025Imagine Beyond! Distributionally Robust Auto-Encoding for State Space Coverage in Online Reinforcement Learningcited by 0
  5. 2026Goal-Conditioned Agents that Learn Everything All at Oncecited by 0