mlsanityData to 7 Oct 2026

Imitation Learning from Imperfect Demonstration

arXiv 1901.09387 ↗First posted 27 Jan 2019cs.LG cs.AI stat.MLcited by 112

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning (IL) aims to learn an optimal policy from demonstrations. However, such demonstrations are often imperfect since collecting optimal ones is costly. To effectively learn from imperfect demonstrations, we propose a novel approach that utilizes confidence scores, which describe the quality of demonstrations. More specifically, we propose two confidence-based IL methods, namely two-step importance weighting IL (2IWIL) and generative adversarial IL with imperfect demonstration and confidence (IC-GAIL). We show that confidence scores given only to a small portion of sub-optimal demonstrations significantly improve the performance of IL both theoretically and empirically.

Key facts. Cited by 112 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 14 of its 43 references that are in the map. First posted on arXiv 27 Jan 2019.

Builds on

14 of 43 references.

References · 14

  1. 2015Trust Region Policy Optimizationcited by 2,503
  2. 2016Concrete Problems in AI Safetycited by 1,582
  3. 2016Generative Adversarial Imitation Learningcited by 1,576
  4. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  5. 2017Positive-Unlabeled Learning with Non-Negative Risk Estimatorcited by 217
  6. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  7. 2016Learning Features by Watching Objects Movecited by 138
  8. 2016Learning Motion Patterns in Videoscited by 53
  9. 2017A Deep Reinforcement Learning Chatbotcited by 36
  10. 2017Binary Classification from Positive-Confidence Datacited by 23
  11. 2016Avoiding Wireheading with Value Reinforcement Learningcited by 15
  12. 2017Tracking as Online Decision-Making: Learning a Policy from Streaming Videos with Reinforcement Learningcited by 14
2 more
  1. 2018Inference Aided Reinforcement Learning for Incentive Mechanism Design in Crowdsourcingcited by 7
  2. 2017Learning to Learn from Noisy Web Videoscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 2; 2020: 3; 2021: 18; 2022: 25; 2023: 23; 2024: 12; 2025: 14; 2026: 15.

112 papers.

Sort:

By area:

  1. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  2. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  3. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrationscited by 89
  4. 2022Reinforcement Learning with Sparse Rewards using Guidance from Offline Demonstrationcited by 54
  5. 2022Learning Generalizable Dexterous Manipulation from Human Grasp Affordancecited by 46
  6. 2021Confidence-Aware Imitation Learning from Demonstrations with Varying Optimalitycited by 44
  7. 2021Learning Multimodal Rewards from Rankingscited by 41
  8. 2023Query-Dependent Prompt Evaluation and Optimization with Offline Inverse RLcited by 37
  9. 2025Planning with Reasoning using Vision Language World Modelcited by 36
  10. 2021JueWu-MC: Playing Minecraft with Sample-efficient Hierarchical Reinforcement Learningcited by 28
  11. 2020Robust Imitation Learning from Noisy Demonstrationscited by 26
  12. 2021Learning from Imperfect Demonstrations from Agents with Varying Dynamicscited by 26
38 more
  1. 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
  2. 2023Offline Imitation Learning with Suboptimal Demonstrations via Relaxed Distribution Matchingcited by 24
  3. 2023Towards Robust Offline Reinforcement Learning under Diverse Data Corruptioncited by 24
  4. 2021Causal Navigation by Continuous-time Neural Networkscited by 23
  5. 2022You Only Live Once: Single-Life Reinforcement Learningcited by 23
  6. 2021Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racingcited by 22
  7. 2026PALM: Progress-Aware Policy Learning via Affordance Reasoning for Long-Horizon Robotic Manipulationcited by 20
  8. 2022Benchmarking Constraint Inference in Inverse Reinforcement Learningcited by 18
  9. 2023Chain-of-Thought Predictive Controlcited by 18
  10. 2023MEGA-DAgger: Imitation Learning with Multiple Imperfect Expertscited by 17
  11. 2022SelfD: Self-Learning Large-Scale Driving Policies From the Webcited by 16
  12. 2025Curating Demonstrations using Online Experiencecited by 16
  13. 2025SAIL: Faster-than-Demonstration Execution of Imitation Learning Policiescited by 16
  14. 2023Reward Finetuning for Faster and More Accurate Unsupervised Object Discoverycited by 14
  15. 2025Generative Models in Decision Making: A Surveycited by 14
  16. 2024Learning from Imperfect Demonstrations with Self-Supervision for Robotic Manipulationcited by 13
  17. 2019VILD: Variational Imitation Learning with Diverse-quality Demonstrationscited by 12
  18. 2024Inverse Reinforcement Learning by Estimating Expertise of Demonstratorscited by 11
  19. 2024How to Leverage Diverse Demonstrations in Offline Imitation Learningcited by 11
  20. 2020Episodic Self-Imitation Learning with Hindsightcited by 10
  21. 2021Learning Feasibility to Imitate Demonstrators with Different Dynamicscited by 10
  22. 2022Robust Imitation Learning from Corrupted Demonstrationscited by 10
  23. 2022Receding Horizon Inverse Reinforcement Learningcited by 10
  24. 2022Don't Copy the Teacher: Data and Model Challenges in Embodied Dialoguecited by 10
  25. 2020Driving Through Ghosts: Behavioral Cloning with False Positivescited by 8
  26. 2021Learning from an Exploring Demonstrator: Optimal Reward Estimation for Banditscited by 8
  27. 2022Disturbance-Injected Robust Imitation Learning with Task Achievementcited by 8
  28. 2023CLIP-Motion: Learning Reward Functions for Robotic Actions Using Consecutive Observationscited by 8
  29. 2022Goal-Aware Generative Adversarial Imitation Learning from Imperfect Demonstration for Robotic Cloth Manipulationcited by 7
  30. 2024SPRINQL: Sub-optimal Demonstrations driven Offline Imitation Learningcited by 7
  31. 2023GO-DICE: Goal-Conditioned Option-Aware Offline Imitation Learning via Stationary Distribution Correction Estimationcited by 6
  32. 2019Better-than-Demonstrator Imitation Learning via Automatically-Ranked Demonstrationscited by 5
  33. 2021No Need for Interactions: Robust Model-Based Imitation Learning using Neural ODEcited by 5
  34. 2021A Closer Look at Advantage-Filtered Behavioral Cloning in High-Noise Datasetscited by 5
  35. 2022Learning from Imperfect Demonstrations via Adversarial Confidence Transfercited by 5
  36. 2022Are All Vision Models Created Equal? A Study of the Open-Loop to Closed-Loop Causality Gapcited by 5
  37. 2023Skill Disentanglement for Imitation Learning from Suboptimal Demonstrationscited by 5
  38. 2023Learning Models of Adversarial Agent Behavior under Partial Observabilitycited by 5

50 most cited of 112 All 112 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021Robust Maximum Entropy Behavior Cloning
  2. 2020The MAGICAL Benchmark for Robust Imitation
  3. 2023Aligning Human Intent from Imperfect Demonstrations with Confidence-based Inverse soft-Q Learning
  4. 2022Robust Learning from Observation with Model Misspecification
  5. 2024Imitating from auxiliary imperfect demonstrations via Adversarial Density Weighted Regression
  6. 2018Risk-Sensitive Generative Adversarial Imitation Learning
  7. 2022Robust Imitation of a Few Demonstrations with a Backwards Model
  8. 2020f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning