mlsanityData to 7 Oct 2026

Robust Imitation Learning against Variations in Environment Dynamics

arXiv 2206.09314 ↗First posted 19 Jun 2022cs.LG cs.AIcited by 26

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

In this paper, we propose a robust imitation learning (IL) framework that improves the robustness of IL when environment dynamics are perturbed. The existing IL framework trained in a single environment can catastrophically fail with perturbations in environment dynamics because it does not capture the situation that underlying environment dynamics can be changed. Our framework effectively deals with environments with varying dynamics by imitating multiple experts in sampled environment dynamics to enhance the robustness in general variations in environment dynamics. In order to robustly imitate the multiple sample experts, we minimize the risk with respect to the Jensen-Shannon divergence between the agent's policy and each of the sample experts. Numerical results show that our algorithm significantly improves robustness against dynamics perturbations compared to conventional IL baselines.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 39 of its 52 references that are in the map. First posted on arXiv 19 Jun 2022.

Builds on

39 of 52 references.

Closest ancestors · 12

Same direction, newest first

  1. 2021Cross-domain Imitation from Observations20 May
  2. 2020The MAGICAL Benchmark for Robust Imitation1 Nov
  3. 2020Robust Imitation Learning from Noisy Demonstrations20 Oct
  4. 2020ADAIL: Adaptive Adversarial Imitation Learning23 Aug
  5. 2020State-only Imitation with Transition Dynamics Mismatch27 Feb
  6. 2020Multi-Agent Interactions Modeling with Correlated Policies4 Jan
  7. 2019State Alignment-based Imitation Learning21 Nov
  8. 2019Watch, Try, Learn: Meta-Learning from Demonstrations and Reward7 Jun
  9. 2019Imitation Learning from Imperfect Demonstration27 Jan
  10. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep
  11. 2018Generative Adversarial Imitation from Observation17 Jul
  12. 2018Behavioral Cloning from Observation4 May

Other references · 27

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  6. 2017Improved Training of Wasserstein GANscited by 2,223
  7. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  8. 2016Generative Adversarial Imitation Learningcited by 1,576
  9. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  10. 2017Robust Adversarial Reinforcement Learningcited by 439
  11. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  12. 2017One-Shot Imitation Learningcited by 232
15 more
  1. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  2. 2016Coupled Generative Adversarial Networkscited by 216
  3. 2017DART: Noise Injection for Robust Imitation Learningcited by 187
  4. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 167
  5. 2017Robust Imitation of Diverse Behaviorscited by 85
  6. 2019Robust Reinforcement Learning for Continuous Control with Model Misspecificationcited by 76
  7. 2018Task-Embedded Control Networks for Few-Shot Imitation Learningcited by 65
  8. 2020Robust Reinforcement Learning using Adversarial Populationscited by 64
  9. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variablescited by 50
  10. 2020Experience Replay with Likelihood-free Importance Weightscited by 42
  11. 2018Soft-Robust Actor-Critic Policy-Gradientcited by 37
  12. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learningcited by 36
  13. 2020Robust Inverse Reinforcement Learning under Transition Dynamics Mismatchcited by 33
  14. 2018Multi-task Maximum Entropy Inverse Reinforcement Learningcited by 20
  15. 2018Learning Robust Optionscited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 4; 2024: 4; 2025: 8; 2026: 8.

26 papers.

Sort:

By area:

  1. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  2. 2024Contrastive Representation for Data Filtering in Cross-Domain Offline Reinforcement Learningcited by 18
  3. 2023Policy Gradient Algorithms for Robust MDPs with Non-Rectangular Uncertainty Setscited by 15
  4. 2023Domain Adaptive Imitation Learning with Visual Observationcited by 11
  5. 2023Robust Visual Imitation Learning with Inverse Dynamics Representationscited by 8
  6. 2024Exclusively Penalized Q-learning for Offline Reinforcement Learningcited by 8
  7. 2025Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuningcited by 8
  8. 2025Wolfpack Adversarial Attack for Robust Multi-Agent Reinforcement Learningcited by 6
  9. 2026Flow Actor-Critic for Offline Reinforcement Learningcited by 6
  10. 2025Self-Improving Skill Learning for Robust Skill-based Meta-Reinforcement Learningcited by 5
  11. 2022Environment Design for Inverse Reinforcement Learningcited by 4
  12. 2025Task-Aware Virtual Training: Enhancing Generalization in Meta-Reinforcement Learning for Out-of-Distribution Taskscited by 3
14 more
  1. 2024DIDA: Denoised Imitation Learning based on Domain Adaptationcited by 2
  2. 2025Efficient Cross-Domain Offline Reinforcement Learning with Dynamics- and Value-Aligned Data Filteringcited by 2
  3. 2025Inversely Learning Transferable Rewards via Abstracted Statescited by 1
  4. 2025Balance Equation-based Distributionally Robust Offline Imitation Learningcited by 1
  5. 2025Distributionally Robust Imitation Learning: Layered Control Architecture for Certifiable Autonomycited by 1
  6. 2026Interaction-Breaking Adversarial Learning Framework for Robust Multi-Agent Reinforcement Learningcited by 1
  7. 2022Cross apprenticeship learning framework: Properties and solution approachescited by 0
  8. 2024DyPNIPP: Predicting Environment Dynamics for RL-based Robust Informative Path Planningcited by 0
  9. 2026Bridging Domain Gaps with Target-Aligned Generation for Offline Reinforcement Learningcited by 0
  10. 2026When Dynamics Shift, Robust Task Inference Wins: Offline Imitation Learning with Behavior Foundation Models Revisitedcited by 0
  11. 2026ConTraIRL: Factorized Contrastive Abstractions for Transferable IRLcited by 0
  12. 2026Distributionally Robust and Safe Imitation Learningcited by 0
  13. 2026Scalable Causal Imitation Learningcited by 0
  14. 2026Generalize and Guide: Decomposing Rewards for Few-Shot Inverse Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Robust Learning from Observation with Model Misspecification
  2. 2022Robust Imitation of a Few Demonstrations with a Backwards Model
  3. 2025RISE: Robust Imitation through Stochastic Encoding
  4. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitions
  5. 2024Reward-free World Models for Online Imitation Learning
  6. 2024Efficient Imitation Learning with Conservative World Models
  7. 2025A Model-Based Approach to Imitation Learning through Multi-Step Predictions
  8. 2022Imitation Learning by State-Only Distribution Matching