mlsanityData to 7 Oct 2026

Learning from Imperfect Demonstrations from Agents with Varying Dynamics

arXiv 2103.05910 ↗First posted 10 Mar 2021cs.LG cs.ROcited by 26

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning enables robots to learn from demonstrations. Previous imitation learning algorithms usually assume access to optimal expert demonstrations. However, in many real-world applications, this assumption is limiting. Most collected demonstrations are not optimal or are produced by an agent with slightly different dynamics. We therefore address the problem of imitation learning when the demonstrations can be sub-optimal or be drawn from agents with varying dynamics. We develop a metric composed of a feasibility score and an optimality score to measure how useful a demonstration is for imitation learning. The proposed score enables learning from more informative demonstrations, and disregarding the less relevant demonstrations. Our experiments on four environments in simulation and on a real robot show improved learned policies with higher expected return.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 20 of its 41 references that are in the map. First posted on arXiv 10 Mar 2021.

Builds on

20 of 41 references.

Closest ancestors · 9

Same direction, newest first

  1. 2020Learning Sparse Rewarded Tasks from Sub-Optimal Demonstrations1 Apr
  2. 2019Continuous Online Learning and New Insights to Online Imitation Learning3 Dec
  3. 2019State Alignment-based Imitation Learning21 Nov
  4. 2019Wasserstein Adversarial Imitation Learning19 Jun
  5. 2019Imitation Learning as f-Divergence Minimization30 May
  6. 2019Provably Efficient Imitation Learning from Observation Alone27 May
  7. 2019Imitation Learning from Imperfect Demonstration27 Jan
  8. 2018Generative Adversarial Imitation from Observation17 Jul
  9. 2018Behavioral Cloning from Observation4 May

Other references · 11

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Trust Region Policy Optimizationcited by 2,503
  3. 2016OpenAI Gymcited by 1,684
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  6. 2018Importance Weighted Adversarial Nets for Partial Domain Adaptationcited by 90
  7. 2018Adversarial Imitation via Variational Inverse Reinforcement Learningcited by 36
  8. 2020When Humans Aren't Optimal: Robots that Collaborate with Risk-Aware Humanscited by 34
  9. 2019Controlling Assistive Robots with Learned Latent Actionscited by 28
  10. 2020Reinforcement Learning based Control of Imitative Policies for Near-Accident Drivingcited by 23
  11. 2016A First Look at Ad-block Detection: A New Arms Race on the Webcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 9; 2023: 4; 2024: 3; 2025: 5; 2026: 3.

26 papers.

Sort:

By area:

  1. 2022Imitation Learning by Estimating Expertise of Demonstratorscited by 50
  2. 2022Learning Generalizable Dexterous Manipulation from Human Grasp Affordancecited by 46
  3. 2021Confidence-Aware Imitation Learning from Demonstrations with Varying Optimalitycited by 44
  4. 2022The Expertise Problem: Learning from Specialized Feedbackcited by 20
  5. 2023MAHALO: Unifying Offline Reinforcement Learning and Imitation Learning from Observationscited by 18
  6. 2023Distance Weighted Supervised Learning for Offline Interaction Datacited by 18
  7. 2024A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agentscited by 16
  8. 2024Inverse Reinforcement Learning by Estimating Expertise of Demonstratorscited by 11
  9. 2021Learning Feasibility to Imitate Demonstrators with Different Dynamicscited by 10
  10. 2023Cross-Episodic Curriculum for Transformer Agentscited by 10
  11. 2022Learning from Imperfect Demonstrations via Adversarial Confidence Transfercited by 5
  12. 2022Learning Preferences for Interactive Autonomycited by 5
14 more
  1. 2025IL-SOAR : Imitation Learning with Soft Optimistic Actor cRiticcited by 5
  2. 2022Reinforcement learning with Demonstrations from Mismatched Task under Sparse Rewardcited by 4
  3. 2024Select to Perfect: Imitating desired behavior from large multi-agent datacited by 4
  4. 2022Out-of-Dynamics Imitation Learning from Multimodal Demonstrationscited by 3
  5. 2025Contrastive Learning from Exploratory Actions: Leveraging Natural Interactions for Preference Elicitationcited by 3
  6. 2022Robust Offline Reinforcement Learning with Gradient Penalty and Constraint Relaxationcited by 2
  7. 2026InterPReT: Interactive Policy Restructuring and Training Enable Effective Imitation Learning from Laypersonscited by 2
  8. 2025Feasibility-aware Imitation Learning from Observations through a Hand-mounted Demonstration Interfacecited by 1
  9. 2025Flexible Locomotion Learning with Diffusion Model Predictive Controlcited by 1
  10. 2022Task-Agnostic Learning to Accomplish New Taskscited by 0
  11. 2023Divide and Repair: Using Options to Improve Performance of Imitation Learning Against Adversarial Demonstrationscited by 0
  12. 2025Learning from Observation: A Survey of Recent Advancescited by 0
  13. 2026Feasibility-aware Imitation Learning from Observation with Multimodal Feedbackcited by 0
  14. 2026Robo-ValueRL: Reliable Value Estimation for Offline-to-Online Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Dual Approach to Imitation Learning from Observations with Offline Datasets
  2. 2022Robust Imitation of a Few Demonstrations with a Backwards Model
  3. 2018An Algorithmic Perspective on Imitation Learning
  4. 2025When a Robot is More Capable than a Human: Learning from Constrained Demonstrators
  5. 2021Imitation Learning: Progress, Taxonomies and Challenges
  6. 2023Skill Disentanglement for Imitation Learning from Suboptimal Demonstrations
  7. 2021Learning Online from Corrective Feedback: A Meta-Algorithm for Robotics
  8. 2019Watch, Try, Learn: Meta-Learning from Demonstrations and Reward