mlsanityData to 7 Oct 2026

State Alignment-based Imitation Learning

arXiv 1911.10947 ↗First posted 21 Nov 2019cs.LG stat.MLcited by 73

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Consider an imitation learning problem that the imitator and the expert have different dynamics models. Most of the current imitation learning methods fail because they focus on imitating actions. We propose a novel state alignment-based imitation learning method to train the imitator to follow the state sequences in expert demonstrations as much as possible. The state alignment comes from both local and global perspectives and we combine them into a reinforcement learning framework by a regularized policy update objective. We show the superiority of our method on standard imitation learning settings and imitation learning settings where the expert and imitator have different dynamics models.

Key facts. Cited by 73 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 26 of its 46 references that are in the map. First posted on arXiv 21 Nov 2019.

Builds on

26 of 46 references.

Closest ancestors · 7

Same direction, newest first

  1. 2019Wasserstein Adversarial Imitation Learning19 Jun
  2. 2019Imitation Learning as f-Divergence Minimization30 May
  3. 2019Provably Efficient Imitation Learning from Observation Alone27 May
  4. 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets6 Sep
  5. 2018Generative Adversarial Imitation from Observation17 Jul
  6. 2018Imitating Latent Policies from Observation21 May
  7. 2018Behavioral Cloning from Observation4 May

Other references · 19

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2017Improved Training of Wasserstein GANscited by 2,223
  6. 2016Generative Adversarial Imitation Learningcited by 1,576
  7. 2017Wasserstein GANcited by 635
  8. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 504
  9. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  10. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  11. 2019Efficient Exploration via State Marginal Matchingcited by 186
  12. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
7 more
  1. 2018Zero-Shot Visual Imitationcited by 114
  2. 2018Near-Optimal Representation Learning for Hierarchical Reinforcement Learningcited by 107
  3. 2018Playing hard exploration games by watching YouTubecited by 98
  4. 2017Coordinated Multi-Agent Imitation Learningcited by 70
  5. 2018Observe and Look Further: Achieving Consistent Performance on Ataricited by 42
  6. 2018Adversarial Imitation via Variational Inverse Reinforcement Learningcited by 36
  7. 2018Optimal Transport on Discrete Domainscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 7; 2021: 11; 2022: 21; 2023: 11; 2024: 13; 2025: 9; 2026: 1.

73 papers.

Sort:

By area:

  1. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 194
  2. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
  3. 2020Primal Wasserstein Imitation Learningcited by 120
  4. 2021XIRL: Cross-embodiment Inverse Reinforcement Learningcited by 101
  5. 2023UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learningcited by 101
  6. 2020State-Only Imitation Learning for Dexterous Manipulationcited by 90
  7. 2021Off-Policy Imitation Learning from Observationscited by 70
  8. 2020f-IRL: Inverse Reinforcement Learning via State Marginal Matchingcited by 67
  9. 2020Strictly Batch Imitation Learning by Energy-based Distribution Matchingcited by 54
  10. 2021Cross-Domain Imitation Learning via Optimal Transportcited by 49
  11. 2023Zero-Shot Robot Manipulation from Passive Human Videoscited by 49
  12. 2022Learning Generalizable Dexterous Manipulation from Human Grasp Affordancecited by 46
38 more
  1. 2022Versatile Offline Imitation from Observations and Examples via Regularized State-Occupancy Matchingcited by 43
  2. 2022Maximum-Likelihood Inverse Reinforcement Learning with Finite-Time Guaranteescited by 42
  3. 2020Energy-Based Imitation Learningcited by 39
  4. 2025CLAM: Continuous Latent Action Models for Robot Learning from Unlabeled Demonstrationscited by 39
  5. 2020Robust Inverse Reinforcement Learning under Transition Dynamics Mismatchcited by 33
  6. 2023Cross-Domain Policy Adaptation via Value-Guided Data Filteringcited by 30
  7. 2021Learning from Imperfect Demonstrations from Agents with Varying Dynamicscited by 26
  8. 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
  9. 2023State Regularized Policy Optimization on Data with Dynamics Shiftcited by 23
  10. 2022LobsDICE: Offline Learning from Observation via Stationary Distribution Correction Estimationcited by 22
  11. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  12. 2022A Ranking Game for Imitation Learningcited by 18
  13. 2023MAHALO: Unifying Offline Reinforcement Learning and Imitation Learning from Observationscited by 18
  14. 2024PhysReaction: Physically Plausible Real-Time Humanoid Reaction Synthesis via Forward Dynamics Guided 4D Imitationcited by 18
  15. 2024A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agentscited by 16
  16. 2021Unsupervised Domain Adaptation with Dynamics-Aware Rewards in Reinforcement Learningcited by 15
  17. 2022Learn what matters: cross-domain imitation learning with task-relevant embeddingscited by 14
  18. 2022REvolveR: Continuous Evolutionary Models for Robot-to-robot Policy Transfercited by 12
  19. 2022Plan Your Target and Learn Your Skills: Transferable State-Only Imitation Learning via Decoupled Policy Optimizationcited by 12
  20. 2022Structural Estimation of Markov Decision Processes in High-Dimensional State Space with Finite-Time Guaranteescited by 12
  21. 2023CEIL: Generalized Contextual Imitation Learningcited by 12
  22. 2022Imitation Learning from Observations under Transition Model Disparitycited by 11
  23. 2023Domain Adaptive Imitation Learning with Visual Observationcited by 11
  24. 2021Learning Feasibility to Imitate Demonstrators with Different Dynamicscited by 10
  25. 2022Robust Learning from Observation with Model Misspecificationcited by 9
  26. 2024DrS: Learning Reusable Dense Rewards for Multi-Stage Taskscited by 9
  27. 2024xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editingcited by 9
  28. 2022HERD: Continuous Human-to-Robot Evolution for Learning from Human Demonstrationcited by 8
  29. 2022Masked Imitation Learning: Discovering Environment-Invariant Modalities in Multimodal Demonstrationscited by 6
  30. 2021Skeletal Feature Compensation for Imitation Learning with Embodiment Mismatchcited by 5
  31. 2022Transferable Reward Learning by Dynamics-Agnostic Discriminator Ensemblecited by 5
  32. 2023Offline Imitation from Observation via Primal Wasserstein State Occupancy Matchingcited by 5
  33. 2025IL-SOAR : Imitation Learning with Soft Optimistic Actor cRiticcited by 5
  34. 2022Reinforcement learning with Demonstrations from Mismatched Task under Sparse Rewardcited by 4
  35. 2021Off-Dynamics Inverse Reinforcement Learning from Hetero-Domaincited by 3
  36. 2022Co-Imitation: Learning Design and Behaviour by Imitationcited by 3
  37. 2022Out-of-Dynamics Imitation Learning from Multimodal Demonstrationscited by 3
  38. 2024Meta-Evolve: Continuous Robot Evolution for One-to-many Policy Transfercited by 3

50 most cited of 73 All 73 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2020State-only Imitation with Transition Dynamics Mismatch
  2. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances
  3. 2025A Model-Based Approach to Imitation Learning through Multi-Step Predictions
  4. 2021Cross-domain Imitation from Observations
  5. 2024Rethinking Inverse Reinforcement Learning: from Data Alignment to Task Alignment
  6. 2023Robust Visual Imitation Learning with Inverse Dynamics Representations
  7. 2024Efficient Imitation Learning with Conservative World Models
  8. 2020Interactive Imitation Learning in State-Space