mlsanityData to 7 Oct 2026

Goal-conditioned Imitation Learning

arXiv 1906.05838 ↗First posted 13 Jun 2019cs.LG cs.AI cs.NEcited by 159

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Designing rewards for Reinforcement Learning (RL) is challenging because it needs to convey the desired task, be efficient to optimize, and be easy to compute. The latter is particularly problematic when applying RL to robotics, where detecting whether the desired configuration is reached might require considerable supervision and instrumentation. Furthermore, we are often interested in being able to reach a wide range of configurations, hence setting up a different reward every time might be unpractical. Methods like Hindsight Experience Replay (HER) have recently shown promise to learn policies able to reach many goals, without the need of a reward. Unfortunately, without tricks like resetting to points along the trajectory, HER might require many samples to discover how to reach certain areas of the state-space. In this work we investigate different approaches to incorporate demonstrations to drastically speed up the convergence to a policy able to reach any goal, also surpassing the performance of an agent trained with other Imitation Learning algorithms. Furthermore, we show our method can also be used when the available expert trajectories do not contain the actions, which can leverage kinesthetic or third person demonstration. The code is available at https://sites.google.com/view/goalconditioned-il/.

Key facts. Cited by 159 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 29 of its 44 references that are in the map. First posted on arXiv 13 Jun 2019.

Builds on

29 of 44 references.

Closest ancestors · 5

Same direction, newest first

  1. 2019Generative predecessor models for sample-efficient imitation learning1 Apr
  2. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep
  3. 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets6 Sep
  4. 2018Truncated Horizon Policy Search: Combining Reinforcement Learning & Imitation Learning29 May
  5. 2018Reinforcement Learning from Imperfect Demonstrations14 Feb

Other references · 24

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2017Hindsight Experience Replaycited by 1,096
  6. 2016End to End Learning for Self-Driving Carscited by 1,000
  7. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  8. 2018Learning Dexterous In-Hand Manipulationcited by 643
  9. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  10. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  11. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  12. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
12 more
  1. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  2. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  3. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 185
  4. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 164
  5. 2017Emergence of Locomotion Behaviours in Rich Environmentscited by 159
  6. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 153
  7. 2017Stochastic Neural Networks for Hierarchical Reinforcement Learningcited by 124
  8. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 124
  9. 2017Third-Person Imitation Learningcited by 103
  10. 2018Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Costcited by 79
  11. 2019Self-supervised Learning of Image Embedding for Continuous Controlcited by 16
  12. 2019Adaptive Variance for Changing Sparse-Reward Environmentscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 7; 2021: 16; 2022: 23; 2023: 38; 2024: 29; 2025: 33; 2026: 13.

159 papers.

Sort:

By area:

  1. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  2. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 278
  3. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  4. 2022Interactive Language: Talking to Robots in Real Timecited by 217
  5. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
  6. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 184
  7. 2023Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planningcited by 161
  8. 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 159
  9. 2020Language-Conditioned Imitation Learning for Robot Manipulation Taskscited by 136
  10. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 133
  11. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  12. 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 118
38 more
  1. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RLcited by 89
  2. 2024Foundation Policies with Hilbert Representationscited by 89
  3. 2021Imitation Learning: Progress, Taxonomies and Challengescited by 56
  4. 2023Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasetscited by 56
  5. 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 51
  6. 2022Imitation Learning by Estimating Expertise of Demonstratorscited by 50
  7. 2022Hierarchical Planning Through Goal-Conditioned Offline Reinforcement Learningcited by 48
  8. 2023PlayFusion: Skill Acquisition via Diffusion from Language-Annotated Playcited by 48
  9. 2024Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformerscited by 47
  10. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Drivingcited by 43
  11. 2023GROOT: Learning to Follow Instructions by Watching Gameplay Videoscited by 42
  12. 2024RT-Sketch: Goal-Conditioned Imitation Learning from Hand-Drawn Sketchescited by 41
  13. 2021Adversarial Intrinsic Motivation for Reinforcement Learningcited by 40
  14. 2023Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Predictioncited by 39
  15. 2025DynaGuide: Steering Diffusion Polices with Active Dynamic Guidancecited by 39
  16. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 35
  17. 2023Contrastive Difference Predictive Codingcited by 35
  18. 2024Inference-Time Policy Steering through Human Interactionscited by 35
  19. 2022Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Spacecited by 31
  20. 2022Chain of Thought Imitation with Procedure Cloningcited by 30
  21. 2023Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitationcited by 30
  22. 2025Digital Twin Catalog: A Large-Scale Photorealistic 3D Object Digital Twin Datasetcited by 29
  23. 2025BEAST: Efficient Tokenization of B-Splines Encoded Action Sequences for Imitation Learningcited by 25
  24. 2022Generalization with Lossy Affordances: Leveraging Broad Offline Data for Learning Visuomotor Taskscited by 24
  25. 2022UniMASK: Unified Inference in Sequential Decision Problemscited by 23
  26. 2023Universal Visual Decomposer: Long-Horizon Manipulation Made Easycited by 23
  27. 2020Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimizationcited by 22
  28. 2023Waypoint Transformer: Reinforcement Learning via Supervised Learning with Intermediate Targetscited by 22
  29. 2025Option-aware Temporally Abstracted Value for Offline Goal-Conditioned Reinforcement Learningcited by 22
  30. 2021Continuous Control with Action Quantization from Demonstrationscited by 21
  31. 2022Goal-conditioned dual-action imitation learning for dexterous dual-arm robot manipulationcited by 21
  32. 2022Know Your Boundaries: The Necessity of Explicit Behavioral Cloning in Offline RLcited by 21
  33. 2023Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robotcited by 21
  34. 2023SMORE: Score Models for Offline Goal-Conditioned Reinforcement Learningcited by 21
  35. 2020Inverse Reinforcement Learning with Natural Language Goalscited by 20
  36. 2022Imitating Past Successes can be Very Suboptimalcited by 20
  37. 2024RoboUniView: Visual-Language Model with Unified View Representation for Robotic Manipulationcited by 20
  38. 2024A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoalscited by 20

50 most cited of 159 All 159 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2019Learning to Reach Goals via Iterated Supervised Learning
  2. 2023Backward Learning for Goal-Conditioned Policies
  3. 2019Reward-Conditioned Policies
  4. 2022GoalsEye: Learning High Speed Precision Table Tennis on a Physical Robot
  5. 2019Learning from Trajectories via Subgoal Discovery
  6. 2024Expert Proximity as Surrogate Rewards for Single Demonstration Imitation Learning
  7. 2026Learning from Demonstrations via Capability-Aware Goal Sampling
  8. 2019Watch, Try, Learn: Meta-Learning from Demonstrations and Reward