mlsanityData to 7 Oct 2026

Automatic Goal Generation for Reinforcement Learning Agents

arXiv 1705.06366 ↗First posted 17 May 2017cs.LG cs.AI cs.ROcited by 252

Not on the map. Why?

Cited by 20 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Reinforcement learning is a powerful technique to train an agent to perform a task. However, an agent that is trained using reinforcement learning is only capable of achieving the single task that is specified via its reward function. Such an approach does not scale well to settings in which an agent needs to perform a diverse set of tasks, such as navigating to varying positions in a room or moving objects to varying locations. Instead, we propose a method that allows an agent to automatically discover the range of tasks that it is capable of performing. We use a generator network to propose tasks for the agent to try to achieve, specified as goal states. The generator network is optimized using adversarial training to produce tasks that are always at the appropriate level of difficulty for the agent. Our method thus automatically produces a curriculum of tasks for the agent to learn. We show that, by using this framework, an agent can efficiently and automatically learn to perform a wide set of tasks without requiring any prior knowledge of its environment. Our method can also learn to achieve tasks with sparse rewards, which traditionally pose significant challenges.

Key facts. Cited by 252 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 52; 2022: 51; 2023: 30; 2024: 35; 2025: 46; 2026: 36.

252 papers.

Sort:

By area:

  1. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
  2. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 323
  3. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  4. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  5. 2022Online Decision Transformercited by 194
  6. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  7. 2022Evolving Curricula with Regret-Based Environment Designcited by 166
  8. 2019Goal-conditioned Imitation Learningcited by 159
  9. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
  10. 2021Discovering and Achieving Goals via World Modelscited by 131
  11. 2021Replay-Guided Adversarial Environment Designcited by 108
  12. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 107
38 more
  1. 2025Self-Evolving Curriculum for LLM Reasoningcited by 107
  2. 2024A Survey on Large Language Model-Based Game Agentscited by 106
  3. 2025Online Difficulty Filtering for Reasoning Oriented Reinforcement Learningcited by 100
  4. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RLcited by 89
  5. 2025Efficient Reinforcement Finetuning via Adaptive Curriculum Learningcited by 88
  6. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 71
  7. 2022BYOL-Explore: Exploration by Bootstrapped Predictioncited by 71
  8. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  9. 2023OMNI: Open-endedness via Models of human Notions of Interestingnesscited by 66
  10. 2022A Survey on Model-based Reinforcement Learningcited by 65
  11. 2024Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learningcited by 63
  12. 2022A Policy-Guided Imitation Approach for Offline Reinforcement Learningcited by 58
  13. 2022P3IV: Probabilistic Procedure Planning from Instructional Videos with Weak Supervisioncited by 52
  14. 2025Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replaycited by 52
  15. 2021Adversarially Guided Actor-Criticcited by 47
  16. 2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 41
  17. 2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
  18. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 40
  19. 2021Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraftcited by 34
  20. 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoningcited by 33
  21. 2021Autonomous Reinforcement Learning via Subgoal Curriculacited by 32
  22. 2022Environment Generation for Zero-Shot Compositional Reinforcement Learningcited by 32
  23. 2022LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulationcited by 32
  24. 2021Towards Distraction-Robust Active Visual Trackingcited by 30
  25. 2024No Regrets: Investigating and Improving Regret Approximations for Curriculum Discoverycited by 29
  26. 2022Goal-driven Self-Attentive Recurrent Networks for Trajectory Predictioncited by 28
  27. 2022Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptationcited by 28
  28. 2023Planning Goals for Explorationcited by 28
  29. 2021Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problemscited by 25
  30. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 25
  31. 2021Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Surveycited by 24
  32. 2022Causality-driven Hierarchical Structure Discovery for Reinforcement Learningcited by 23
  33. 2025Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunitiescited by 23
  34. 2021What Can I Do Here? Learning New Skills by Imagining Visual Affordancescited by 22
  35. 2021A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learningcited by 21
  36. 2021Subgoal Search For Complex Reasoning Taskscited by 21
  37. 2021CARL: A Benchmark for Contextual and Adaptive Reinforcement Learningcited by 21
  38. 2020Inverse Reinforcement Learning with Natural Language Goalscited by 20

50 most cited of 252 All 252 (JSON)