mlsanityData to 7 Oct 2026

State-only Imitation with Transition Dynamics Mismatch

arXiv 2002.11879 ↗First posted 27 Feb 2020stat.ML cs.LGcited by 37

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation Learning (IL) is a popular paradigm for training agents to achieve complicated goals by leveraging expert behavior, rather than dealing with the hardships of designing a correct reward function. With the environment modeled as a Markov Decision Process (MDP), most of the existing IL algorithms are contingent on the availability of expert demonstrations in the same MDP as the one in which a new imitator policy is to be learned. This is uncharacteristic of many real-life scenarios where discrepancies between the expert and the imitator MDPs are common, especially in the transition dynamics function. Furthermore, obtaining expert actions may be costly or infeasible, making the recent trend towards state-only IL (where expert demonstrations constitute only states or observations) ever so promising. Building on recent adversarial imitation approaches that are motivated by the idea of divergence minimization, we present a new state-only IL algorithm in this paper. It divides the overall optimization objective into two subproblems by introducing an indirection step and solves the subproblems iteratively. We show that our algorithm is particularly effective when there is a transition dynamics mismatch between the expert and imitator MDPs, while the baseline IL methods suffer from performance degradation. To analyze this, we construct several interesting MDPs by modifying the configuration parameters for the MuJoCo locomotion tasks from OpenAI Gym.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. First posted on arXiv 27 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 6; 2022: 12; 2023: 4; 2024: 5; 2025: 6; 2026: 2.

37 papers.

Sort:

By area:

  1. 2021Off-Policy Imitation Learning from Observationscited by 70
  2. 2022Versatile Offline Imitation from Observations and Examples via Regularized State-Occupancy Matchingcited by 43
  3. 2021Cross-domain Imitation from Observationscited by 42
  4. 2022Maximum-Likelihood Inverse Reinforcement Learning with Finite-Time Guaranteescited by 42
  5. 2020Robust Inverse Reinforcement Learning under Transition Dynamics Mismatchcited by 33
  6. 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
  7. 2023State Regularized Policy Optimization on Data with Dynamics Shiftcited by 23
  8. 2022Identifiability and generalizability from multiple experts in Inverse Reinforcement Learningcited by 16
  9. 2024A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agentscited by 16
  10. 2021Unsupervised Domain Adaptation with Dynamics-Aware Rewards in Reinforcement Learningcited by 15
  11. 2024Off-Dynamics Reinforcement Learning via Domain Adaptation and Reward Augmented Imitationcited by 15
  12. 2022Learn what matters: cross-domain imitation learning with task-relevant embeddingscited by 14
25 more
  1. 2022Structural Estimation of Markov Decision Processes in High-Dimensional State Space with Finite-Time Guaranteescited by 12
  2. 2023CEIL: Generalized Contextual Imitation Learningcited by 12
  3. 2022Imitation Learning from Observations under Transition Model Disparitycited by 11
  4. 2023Domain Adaptive Imitation Learning with Visual Observationcited by 11
  5. 2021Learning Feasibility to Imitate Demonstrators with Different Dynamicscited by 10
  6. 2024Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matchingcited by 10
  7. 2022Robust Learning from Observation with Model Misspecificationcited by 9
  8. 2024xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editingcited by 9
  9. 2020On the Guaranteed Almost Equivalence between Imitation Learning from Observation and Demonstrationcited by 6
  10. 2021Skeletal Feature Compensation for Imitation Learning with Embodiment Mismatchcited by 5
  11. 2022Transferable Reward Learning by Dynamics-Agnostic Discriminator Ensemblecited by 5
  12. 2025IL-SOAR : Imitation Learning with Soft Optimistic Actor cRiticcited by 5
  13. 2022Reinforcement learning with Demonstrations from Mismatched Task under Sparse Rewardcited by 4
  14. 2021Off-Dynamics Inverse Reinforcement Learning from Hetero-Domaincited by 3
  15. 2022Out-of-Dynamics Imitation Learning from Multimodal Demonstrationscited by 3
  16. 2025Action-Constrained Imitation Learningcited by 2
  17. 2022Auto-Encoding Adversarial Imitation Learningcited by 1
  18. 2025Inversely Learning Transferable Rewards via Abstracted Statescited by 1
  19. 2023Policy Optimization with Smooth Guidance Learned from State-Only Demonstrationscited by 0
  20. 2024Rethinking Adversarial Inverse Reinforcement Learning: Policy Imitation, Transferable Reward Recovery and Algebraic Equilibrium Proofcited by 0
  21. 2025On Generalization and Distributional Update for Mimicking Observations with Adequate Explorationcited by 0
  22. 2025Policy Regularization on Globally Accessible States in Cross-Dynamics Reinforcement Learningcited by 0
  23. 2025Learning from Observation: A Survey of Recent Advancescited by 0
  24. 2026Learning When to Stop: Selective Imitation Learning Under Arbitrary Dynamics Shiftcited by 0
  25. 2026ConTraIRL: Factorized Contrastive Abstractions for Transferable IRLcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Imitation Learning by State-Only Distribution Matching
  2. 2022Plan Your Target and Learn Your Skills: Transferable State-Only Imitation Learning via Decoupled Policy Optimization
  3. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitions
  4. 2019State Alignment-based Imitation Learning
  5. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances
  6. 2026Latent Wasserstein Adversarial Imitation Learning
  7. 2023Sample-efficient Adversarial Imitation Learning
  8. 2024Efficient Imitation Learning with Conservative World Models