mlsanityData to 7 Oct 2026

Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement

arXiv 1910.04417 ↗First posted 10 Oct 2019cs.LG cs.AI cs.ROcited by 46

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

This paper studies Learning from Observations (LfO) for imitation learning with access to state-only demonstrations. In contrast to Learning from Demonstration (LfD) that involves both action and state supervision, LfO is more practical in leveraging previously inapplicable resources (e.g. videos), yet more challenging due to the incomplete expert guidance. In this paper, we investigate LfO and its difference with LfD in both theoretical and practical perspectives. We first prove that the gap between LfD and LfO actually lies in the disagreement of inverse dynamics models between the imitator and the expert, if following the modeling approach of GAIL. More importantly, the upper bound of this gap is revealed by a negative causal entropy which can be minimized in a model-free way. We term our method as Inverse-Dynamics-Disagreement-Minimization (IDDM) which enhances the conventional LfO method through further bridging the gap to LfD. Considerable empirical results on challenging benchmarks indicate that our method attains consistent improvements over other LfO counterparts.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 23 of its 46 references that are in the map. First posted on arXiv 10 Oct 2019.

Builds on

23 of 46 references.

Closest ancestors · 5

Same direction, newest first

  1. 2019Provably Efficient Imitation Learning from Observation Alone27 May
  2. 2019Imitation Learning from Video by Leveraging Proprioception22 May
  3. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow1 Oct
  4. 2018Generative Adversarial Imitation from Observation17 Jul
  5. 2018Behavioral Cloning from Observation4 May

Other references · 18

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  4. 2016OpenAI Gymcited by 1,684
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  7. 2016End to End Learning for Self-Driving Carscited by 1,000
  8. 2018MINE: Mutual Information Neural Estimationcited by 680
  9. 2018Learning Dexterous In-Hand Manipulationcited by 643
  10. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 504
  11. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  12. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 361
6 more
  1. 2017One-Shot Imitation Learningcited by 232
  2. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  3. 2018Learning to Walk via Deep Reinforcement Learningcited by 153
  4. 2017Third-Person Imitation Learningcited by 103
  5. 2019Maximum Entropy Generators for Energy-Based Modelscited by 55
  6. 2018Task Transfer by Preference-Based Cost Learningcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 7; 2022: 11; 2023: 11; 2024: 5; 2025: 9; 2026: 1.

46 papers.

Sort:

By area:

  1. 2023Learning to Act without Actionscited by 116
  2. 2021XIRL: Cross-embodiment Inverse Reinforcement Learningcited by 101
  3. 2021Off-Policy Imitation Learning from Observationscited by 70
  4. 2021MobILE: Model-Based Imitation Learning From Observation Alonecited by 44
  5. 2022Versatile Offline Imitation from Observations and Examples via Regularized State-Occupancy Matchingcited by 43
  6. 2021Cross-domain Imitation from Observationscited by 42
  7. 2023GROOT: Learning to Follow Instructions by Watching Gameplay Videoscited by 42
  8. 2025CLAM: Continuous Latent Action Models for Robot Learning from Unlabeled Demonstrationscited by 39
  9. 2023Hyper-Decision Transformer for Efficient Online Policy Adaptationcited by 32
  10. 2022LobsDICE: Offline Learning from Observation via Stationary Distribution Correction Estimationcited by 22
  11. 2023SMORE: Score Models for Offline Goal-Conditioned Reinforcement Learningcited by 21
  12. 2022A Ranking Game for Imitation Learningcited by 18
34 more
  1. 2023MAHALO: Unifying Offline Reinforcement Learning and Imitation Learning from Observationscited by 18
  2. 2023Look Beneath the Surface: Exploiting Fundamental Symmetry for Sample-Efficient Offline RLcited by 18
  3. 2021Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Taskscited by 13
  4. 2022Plan Your Target and Learn Your Skills: Transferable State-Only Imitation Learning via Decoupled Policy Optimizationcited by 12
  5. 2024Diffusion Imitation from Observationcited by 12
  6. 2022Imitation Learning from Observations under Transition Model Disparitycited by 11
  7. 2023Guiding Online Reinforcement Learning with Action-Free Offline Pretrainingcited by 11
  8. 2023Adversarial Imitation Learning from Visual Observations using Latent Informationcited by 10
  9. 2023Imitation Learning from Observation with Automatic Discount Schedulingcited by 10
  10. 2022Robust Learning from Observation with Model Misspecificationcited by 9
  11. 2022Improving Policy Learning via Language Dynamics Distillationcited by 9
  12. 2024Visually Robust Adversarial Imitation Learning from Videos with Contrastive Learningcited by 9
  13. 2024Pre-trained Visual Dynamics Representations for Efficient Policy Learningcited by 8
  14. 2020On the Guaranteed Almost Equivalence between Imitation Learning from Observation and Demonstrationcited by 6
  15. 2022Imitation Learning by State-Only Distribution Matchingcited by 6
  16. 2026GeoWorld: Geometric World Modelscited by 6
  17. 2023Mimicking Better by Matching the Approximate Action Distributioncited by 5
  18. 2025IL-SOAR : Imitation Learning with Soft Optimistic Actor cRiticcited by 5
  19. 2025Adapting a World Model for Trajectory Following in a 3D Gamecited by 5
  20. 2025TimeRewarder: Learning Dense Reward from Passive Videos via Frame-wise Temporal Distancecited by 5
  21. 2020Concurrent Training Improves the Performance of Behavioral Cloning from Observationcited by 2
  22. 2021Off-policy Imitation Learning from Visual Inputscited by 2
  23. 2025Action-Constrained Imitation Learningcited by 2
  24. 2021Sample Efficient Social Navigation Using Inverse Reinforcement Learningcited by 1
  25. 2022Case-Based Inverse Reinforcement Learning Using Temporal Coherencecited by 1
  26. 2022Open-Ended Diverse Solution Discovery with Regulated Behavior Patterns for Cross-Domain Adaptationcited by 1
  27. 2023GAN-MPC: Training Model Predictive Controllers with Parameterized Cost Functions using Demonstrations from Non-identical Expertscited by 1
  28. 2024MGDA: Model-based Goal Data Augmentation for Offline Goal-conditioned Weighted Supervised Learningcited by 1
  29. 2025Beyond Domain Randomization: Event-Inspired Perception for Visually Robust Adversarial Imitation from Videoscited by 1
  30. 2022Unsupervised Reward Shaping for a Robotic Sequential Picking Task from Visual Observations in a Logistics Scenariocited by 0
  31. 2024Representation Alignment from Human Feedback for Cross-Embodiment Reward Learning from Mixed-Quality Demonstrationscited by 0
  32. 2025On Generalization and Distributional Update for Mimicking Observations with Adequate Explorationcited by 0
  33. 2025Learning from Observation: A Survey of Recent Advancescited by 0
  34. 2025When a Robot is More Capable than a Human: Learning from Constrained Demonstratorscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026On the Sample Efficiency of Inverse Dynamics Models for Semi-Supervised Imitation Learning
  2. 2024A Dual Approach to Imitation Learning from Observations with Offline Datasets
  3. 2025Value from Observations: Towards Large-Scale Imitation Learning via Self-Improvement
  4. 2020Causal Transfer for Imitation Learning and Decision Making under Sensor-shift
  5. 2023Imitation Learning from Observation through Optimal Transport
  6. 2026Towards Generalisable Imitation Learning Through Conditioned Transition Estimation and Online Behaviour Alignment
  7. 2022Deconfounding Imitation Learning with Variational Inference
  8. 2019Imitation Learning as f-Divergence Minimization