mlsanityData to 7 Oct 2026

Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learning

arXiv 2007.02832 ↗First posted 6 Jul 2020cs.LG cs.AI cs.ROcited by 100

Not on the map. Why?

Abstract (arXiv)

What goals should a multi-goal reinforcement learning agent pursue during training in long-horizon tasks? When the desired (test time) goal distribution is too distant to offer a useful learning signal, we argue that the agent should not pursue unobtainable goals. Instead, it should set its own intrinsic goals that maximize the entropy of the historical achieved goal distribution. We propose to optimize this objective by having the agent pursue past achieved goals in sparsely explored areas of the goal space, which focuses exploration on the frontier of the achievable goal set. We show that our strategy achieves an order of magnitude better sample efficiency than the prior state of the art on long-horizon multi-goal tasks including maze navigation and block stacking.

Key facts. Cited by 100 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 23; 2023: 23; 2024: 12; 2025: 18; 2026: 11.

100 papers.

Sort:

By area:

  1. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  2. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
  3. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
  4. 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 118
  5. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RLcited by 89
  6. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 80
  7. 2022BYOL-Explore: Exploration by Bootstrapped Predictioncited by 71
  8. 2022A Survey on Model-based Reinforcement Learningcited by 65
  9. 2023Controllability-Aware Unsupervised Skill Discoverycited by 49
  10. 2023Decomposing the Enigma: Subgoal-based Demonstration Learning for Formal Theorem Provingcited by 37
  11. 2023What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL?cited by 36
  12. 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoningcited by 33
38 more
  1. 2023Planning Goals for Explorationcited by 28
  2. 2023Flipping Coins to Estimate Pseudocounts for Exploration in Reinforcement Learningcited by 28
  3. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 25
  4. 2024Grounding Video Models to Actions through Goal Conditioned Explorationcited by 25
  5. 2022Causality-driven Hierarchical Structure Discovery for Reinforcement Learningcited by 23
  6. 2021Subgoal Search For Complex Reasoning Taskscited by 21
  7. 2025Learning to Reason at the Frontier of Learnabilitycited by 20
  8. 2023CFlowNets: Continuous Control with Generative Flow Networkscited by 19
  9. 2021Active Hierarchical Exploration with Stable Subgoal Representation Learningcited by 17
  10. 2024TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representationscited by 17
  11. 2022Understanding the Complexity Gains of Single-Task RL with a Curriculumcited by 16
  12. 2023A Definition of Open-Ended Learning Problems for Goal-Conditioned Agentscited by 16
  13. 2021Learn Goal-Conditioned Policy with Intrinsic Motivation for Deep Reinforcement Learningcited by 15
  14. 2021C-Planning: An Automatic Curriculum for Learning Goal-Reaching Taskscited by 15
  15. 2021MHER: Model-based Hindsight Experience Replaycited by 14
  16. 2021MapGo: Model-Assisted Policy Optimization for Goal-Oriented Taskscited by 13
  17. 2025Normalizing Flows are Capable Models for Continuous Controlcited by 13
  18. 2021Learning Domain Invariant Representations in Goal-conditioned Block MDPscited by 12
  19. 2023CQM: Curriculum Reinforcement Learning with a Quantized World Modelcited by 11
  20. 2025DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learningcited by 11
  21. 2022Generative Planning for Temporally Coordinated Exploration in Reinforcement Learningcited by 10
  22. 2021Action Redundancy in Reinforcement Learningcited by 9
  23. 2022Continual Auxiliary Task Learningcited by 9
  24. 2022Bilinear value networkscited by 9
  25. 2024Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learningcited by 9
  26. 2023Variational Curriculum Reinforcement Learning for Unsupervised Discovery of Skillscited by 8
  27. 2025Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learningcited by 8
  28. 2021DisTop: Discovering a Topological representation to learn diverse and rewarding skillscited by 7
  29. 2023Self-Motivated Multi-Agent Explorationcited by 7
  30. 2023Hierarchical Empowerment: Towards Tractable Empowerment-Based Skill Learningcited by 7
  31. 2025MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spacescited by 7
  32. 2022E-MAPP: Efficient Multi-Agent Reinforcement Learning with Parallel Program Guidancecited by 6
  33. 2023Building Open-Ended Embodied Agent via Language-Policy Bidirectional Adaptationcited by 6
  34. 2024Dr. Strategy: Model-Based Generalist Agents with Strategic Dreamingcited by 6
  35. 2024Learning World Models for Unconstrained Goal Navigationcited by 6
  36. 2025Epistemically-guided forward-backward explorationcited by 6
  37. 2022Stein Variational Goal Generation for adaptive Exploration in Multi-Goal Reinforcement Learningcited by 5
  38. 2022Cell-Free Latent Go-Explorecited by 5

50 most cited of 100 All 100 (JSON)