mlsanityData to 7 Oct 2026

Imitation Learning from Observations under Transition Model Disparity

arXiv 2204.11446 ↗First posted 25 Apr 2022cs.LG stat.MLcited by 11

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Learning to perform tasks by leveraging a dataset of expert observations, also known as imitation learning from observations (ILO), is an important paradigm for learning skills without access to the expert reward function or the expert actions. We consider ILO in the setting where the expert and the learner agents operate in different environments, with the source of the discrepancy being the transition dynamics model. Recent methods for scalable ILO utilize adversarial learning to match the state-transition distributions of the expert and the learner, an approach that becomes challenging when the dynamics are dissimilar. In this work, we propose an algorithm that trains an intermediary policy in the learner environment and uses it as a surrogate expert for the learner. The intermediary policy is learned such that the state transitions generated by it are close to the state transitions in the expert dataset. To derive a practical and scalable algorithm, we employ concepts from prior work on estimating the support of a probability distribution. Experiments using MuJoCo locomotion tasks highlight that our method compares favorably to the baselines for ILO with transition dynamics mismatch.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 27 of its 37 references that are in the map. First posted on arXiv 25 Apr 2022.

Builds on

27 of 37 references.

Closest ancestors · 13

Same direction, newest first

  1. 2021Off-Policy Imitation Learning from Observations25 Feb
  2. 2020Reparameterized Variational Divergence Minimization for Stable Imitation18 Jun
  3. 2020State-only Imitation with Transition Dynamics Mismatch27 Feb
  4. 2019State Alignment-based Imitation Learning21 Nov
  5. 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement10 Oct
  6. 2019Imitation Learning as f-Divergence Minimization30 May
  7. 2019Provably Efficient Imitation Learning from Observation Alone27 May
  8. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation16 May
  9. 2019Reinforced Imitation in Heterogeneous Action Space6 Apr
  10. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow1 Oct
  11. 2018Generative Adversarial Imitation from Observation17 Jul
  12. 2018Imitating Latent Policies from Observation21 May
1 more
  1. 2018Behavioral Cloning from Observation4 May

Other references · 14

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2018Exploration by Random Network Distillationcited by 966
  6. 2016Deep Variational Information Bottleneckcited by 915
  7. 2018Which Training Methods for GANs do actually Converge?cited by 672
  8. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  9. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  10. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  11. 2017Third-Person Imitation Learningcited by 103
  12. 2017Robust Imitation of Diverse Behaviorscited by 85
2 more
  1. 2017Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learningcited by 84
  2. 2017Learning human behaviors from motion capture by adversarial imitationcited by 75

Built on by

Cited by year

Citing papers per year: 2023: 6; 2024: 2; 2025: 3.

11 papers.

  1. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  2. 2024A Survey of Imitation Learning Methods, Environments and Metricscited by 16
  3. 2023Learning from Visual Observation via Offline Pretrained State-to-Go Transformercited by 14
  4. 2023CEIL: Generalized Contextual Imitation Learningcited by 12
  5. 2023Adversarial Imitation Learning from Visual Observations using Latent Informationcited by 10
  6. 2024Visually Robust Adversarial Imitation Learning from Videos with Contrastive Learningcited by 9
  7. 2023Adaptive action supervision in reinforcement learning from real-world multi-agent demonstrationscited by 8
  8. 2025IL-SOAR : Imitation Learning with Soft Optimistic Actor cRiticcited by 5
  9. 2023Policy Optimization with Smooth Guidance Learned from State-Only Demonstrationscited by 0
  10. 2025Domain-Invariant Per-Frame Feature Extraction for Cross-Domain Imitation Learning with Visual Observationscited by 0
  11. 2025Learning from Observation: A Survey of Recent Advancescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Diffusion Imitation from Observation
  2. 2024A Dual Approach to Imitation Learning from Observations with Offline Datasets
  3. 2023Imitation Learning from Observation through Optimal Transport
  4. 2022Imitation Learning by State-Only Distribution Matching
  5. 2022IL-flOw: Imitation Learning from Observation using Normalizing Flows
  6. 2025Value from Observations: Towards Large-Scale Imitation Learning via Self-Improvement
  7. 2021DEALIO: Data-Efficient Adversarial Learning for Imitation from Observation
  8. 2022Robust Learning from Observation with Model Misspecification