mlsanityData to 7 Oct 2026

Goal-Conditioned Reinforcement Learning: Problems and Solutions

arXiv 2201.08299 ↗First posted 20 Jan 2022cs.AI cs.LGcited by 145

Not on the map. Why?

Cited by 28 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Goal-conditioned reinforcement learning (GCRL), related to a set of complex RL problems, trains an agent to achieve different goals under particular scenarios. Compared to the standard RL solutions that learn a policy solely depending on the states or observations, GCRL additionally requires the agent to make decisions according to different goals. In this survey, we provide a comprehensive overview of the challenges and algorithms for GCRL. Firstly, we answer what the basic problems are studied in this field. Then, we explain how goals are represented and present how existing solutions are designed from different points of view. Finally, we make the conclusion and discuss potential future prospects that recent researches focus on.

Key facts. Cited by 145 papers in the map (data through 7 Oct 2026). Builds on 46 of its 76 references that are in the map. First posted on arXiv 20 Jan 2022.

Builds on

46 of 76 references.

References · 46

  1. 2016OpenAI Gymcited by 1,684
  2. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  3. 2017Hindsight Experience Replaycited by 1,096
  4. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  5. 2020dm_control: Software and Tasks for Continuous Controlcited by 368
  6. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  7. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  8. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  9. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  10. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  11. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 188
  12. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 171
34 more
  1. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  2. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  3. 2021Discovering and Achieving Goals via World Modelscited by 131
  4. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
  5. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  6. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 124
  7. 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 118
  8. 2019Training Agents using Upside-Down Reinforcement Learningcited by 106
  9. 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
  10. 2019Reward-Conditioned Policiescited by 101
  11. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 100
  12. 2020Automatic Curriculum Learning through Value Disagreementcited by 86
  13. 2020Learning with AMIGo: Adversarially Motivated Intrinsic Goalscited by 86
  14. 2018The Laplacian in RL: Learning Representations with Efficient Approximationscited by 83
  15. 2019Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generationcited by 83
  16. 2019Keeping Your Distance: Solving Sparse Reward Tasks Using Self-Balancing Shaped Rewardscited by 76
  17. 2020Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Explorationcited by 68
  18. 2020World Model as a Graph: Learning Latent Landmarks for Planningcited by 67
  19. 2019Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discoverycited by 65
  20. 2020Rewriting History with Inverse RL: Hindsight Inference for Policy Improvementcited by 59
  21. 2019Exploration via Hindsight Goal Generationcited by 53
  22. 2020Goal-Aware Prediction: Learning to Model What Matterscited by 45
  23. 2020Model-Based Visual Planning with Self-Supervised Functional Distancescited by 41
  24. 2019Contextual Imagined Goals for Self-Supervised Robotic Learningcited by 40
  25. 2021Adversarial Intrinsic Motivation for Reinforcement Learningcited by 40
  26. 2019Learning from Trajectories via Subgoal Discoverycited by 27
  27. 2021What Can I Do Here? Learning New Skills by Imagining Visual Affordancescited by 22
  28. 2020PlanGAN: Model-based Planning With Sparse Rewards and Multiple Goalscited by 21
  29. 2019ACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement Learningcited by 17
  30. 2020Weakly-Supervised Reinforcement Learning for Controllable Behaviorcited by 17
  31. 2021MapGo: Model-Assisted Policy Optimization for Goal-Oriented Taskscited by 13
  32. 2020Hindsight Expectation Maximization for Goal-conditioned Reinforcement Learningcited by 12
  33. 2020EpidemiOptim: A Toolbox for the Optimization of Control Policies in Epidemiological Modelscited by 11
  34. 2021Imaginary Hindsight Experience Replay: Curious Model-based Learning for Sparse Reward Taskscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 12; 2023: 23; 2024: 35; 2025: 44; 2026: 31.

145 papers.

Sort:

By area:

  1. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 344
  2. 2024Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learningcited by 147
  3. 2023BQ-NCO: Bisimulation Quotienting for Efficient Neural Combinatorial Optimizationcited by 80
  4. 2025AgentEvolver: Towards Efficient Self-Evolving Agent Systemcited by 77
  5. 2022A Survey on Model-based Reinforcement Learningcited by 65
  6. 2023A Survey of Temporal Credit Assignment in Deep Reinforcement Learningcited by 56
  7. 2024Agile But Safe: Learning Collision-Free High-Speed Legged Locomotioncited by 53
  8. 2023A Survey on Transformers in Reinforcement Learningcited by 50
  9. 2023GROOT: Learning to Follow Instructions by Watching Gameplay Videoscited by 42
  10. 2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 41
  11. 2025What Makes a Good Diffusion Planner for Decision Making?cited by 37
  12. 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoningcited by 33
38 more
  1. 2024Autonomous Improvement of Instruction Following Skills via Foundation Modelscited by 32
  2. 2024MaestroMotif: Skill Design from Artificial Intelligence Feedbackcited by 24
  3. 2025Reward Models in Deep Reinforcement Learning: A Surveycited by 24
  4. 2025Option-aware Temporally Abstracted Value for Offline Goal-Conditioned Reinforcement Learningcited by 22
  5. 2023GOPlan: Goal-conditioned Offline Reinforcement Learning by Planning with Learned Modelscited by 20
  6. 2024A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoalscited by 20
  7. 2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RLcited by 20
  8. 2023Large Sequence Models for Sequential Decision-Making: A Surveycited by 19
  9. 2023Grasp Multiple Objects with One Handcited by 18
  10. 2025Act2Goal: From World Model To General Goal-conditioned Policycited by 18
  11. 2024From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systemscited by 17
  12. 2023A Definition of Open-Ended Learning Problems for Goal-Conditioned Agentscited by 16
  13. 2022On the link between conscious function and general intelligence in humans and machinescited by 15
  14. 2022How Crucial is Transformer in Decision Transformer?cited by 14
  15. 2024Goals as Reward-Producing Programscited by 14
  16. 2024Solving Minimum-Cost Reach Avoid using Reinforcement Learningcited by 14
  17. 2025General agents contain world modelscited by 14
  18. 2023Breadcrumbs to the Goal: Goal-Conditioned Exploration from Human-in-the-Loop Feedbackcited by 13
  19. 2023Autonomous Robotic Reinforcement Learning with Asynchronous Human Feedbackcited by 12
  20. 2024Finer Behavioral Foundation Models via Auto-Regressive Features and Advantage Weightingcited by 12
  21. 2025RoboFlamingo-Plus: Fusion of Depth and RGB Perception with Vision-Language Models for Enhanced Robotic Manipulationcited by 12
  22. 2025Flow-Based Single-Step Completion for Efficient and Expressive Policy Learningcited by 12
  23. 2022State Representation Learning for Goal-Conditioned Reinforcement Learningcited by 11
  24. 2026Conditional Diffusion Guidance under Hard Constraint: A Stochastic Analysis Approachcited by 10
  25. 2024Language-Conditioned Offline RL for Multi-Robot Navigationcited by 9
  26. 2024Looking Backward: Retrospective Backward Synthesis for Goal-Conditioned GFlowNetscited by 8
  27. 2025One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learningcited by 8
  28. 2026Abstracting Robot Manipulation Skills via Mixture-of-Experts Diffusion Policiescited by 8
  29. 2025Causal-aware Large Language Models: Enhancing Decision-Making Through Learning, Adapting and Actingcited by 6
  30. 2024PsyDI: Towards a Personalized and Progressively In-depth Chatbot for Psychological Measurementscited by 5
  31. 2025DreamPolicy: A Unified World-model Policy for Scalable Humanoid Locomotioncited by 5
  32. 2025Offline Goal-Conditioned Reinforcement Learning with Projective Quasimetric Planningcited by 5
  33. 2025Advancing Autonomous VLM Agents via Variational Subgoal-Conditioned Reinforcement Learningcited by 4
  34. 2025Provably Correct Automata Embeddings for Optimal Automata-Conditioned Reinforcement Learningcited by 4
  35. 2025Null Counterfactual Factor Interactions for Goal-Conditioned Reinforcement Learningcited by 4
  36. 2025Hyper-GoalNet: Goal-Conditioned Manipulation Policy Learning with HyperNetworkscited by 4
  37. 2026Laplacian Representations for Decision-Time Planningcited by 4
  38. 2023Swapped goal-conditioned offline reinforcement learningcited by 3

50 most cited of 145 All 145 (JSON)