mlsanityData to 7 Oct 2026

Robust Inverse Reinforcement Learning under Transition Dynamics Mismatch

arXiv 2007.01174 ↗First posted 2 Jul 2020cs.LG stat.MLcited by 33

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

We study the inverse reinforcement learning (IRL) problem under a transition dynamics mismatch between the expert and the learner. Specifically, we consider the Maximum Causal Entropy (MCE) IRL learner model and provide a tight upper bound on the learner's performance degradation based on the ℓ1-distance between the transition dynamics of the expert and the learner. Leveraging insights from the Robust RL literature, we propose a robust MCE IRL algorithm, which is a principled approach to help with this mismatch. Finally, we empirically demonstrate the stable performance of our algorithm compared to the standard MCE IRL algorithm under transition dynamics mismatches in both finite and continuous MDP problems.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 29 of its 71 references that are in the map. First posted on arXiv 2 Jul 2020.

Builds on

29 of 71 references.

Closest ancestors · 4

Same direction, newest first

  1. 2019Learner-aware Teaching: Inverse Reinforcement Learning with Preferences and Constraints2 Jun
  2. 2019Interactive Teaching Algorithms for Inverse Reinforcement Learning28 May
  3. 2018Teaching Inverse Reinforcement Learners via Features and Demonstrations21 Oct
  4. 2018Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behavior21 May

Other references · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  7. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  8. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  9. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  10. 2017Robust Adversarial Reinforcement Learningcited by 439
  11. 2019Reinforcement Learning in Healthcare: A Surveycited by 308
  12. 2019Challenges of Real-World Reinforcement Learningcited by 282
13 more
  1. 2019A Theory of Regularized Markov Decision Processescited by 279
  2. 2018Generative Adversarial Imitation from Observationcited by 183
  3. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
  4. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 167
  5. 2019Provably Efficient Imitation Learning from Observation Alonecited by 78
  6. 2019Robust Reinforcement Learning for Continuous Control with Model Misspecificationcited by 76
  7. 2019State Alignment-based Imitation Learningcited by 73
  8. 2020Robust Reinforcement Learning via Adversarial training with Langevin Dynamicscited by 45
  9. 2020State-only Imitation with Transition Dynamics Mismatchcited by 37
  10. 2016Inverse Reinforcement Learning with Simultaneous Estimation of Rewards and Dynamicscited by 24
  11. 2018Dual Policy Iterationcited by 22
  12. 2018Balancing Two-Player Stochastic Games with Soft Q-Learningcited by 5
  13. 2017Deep Robust Kalman Filtercited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 6; 2023: 5; 2024: 10; 2025: 8; 2026: 2.

33 papers.

Sort:

By area:

  1. 2024Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systemscited by 94
  2. 2022Maximum-Likelihood Inverse Reinforcement Learning with Finite-Time Guaranteescited by 42
  3. 2023Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learningcited by 27
  4. 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
  5. 2024ODRL: A Benchmark for Off-Dynamics Reinforcement Learningcited by 25
  6. 2024Cross-Domain Policy Adaptation by Capturing Representation Mismatchcited by 24
  7. 2022Identifiability and generalizability from multiple experts in Inverse Reinforcement Learningcited by 16
  8. 2023Identifiability and Generalizability in Constrained Inverse Reinforcement Learningcited by 16
  9. 2021Unsupervised Domain Adaptation with Dynamics-Aware Rewards in Reinforcement Learningcited by 15
  10. 2023Policy Gradient Algorithms for Robust MDPs with Non-Rectangular Uncertainty Setscited by 15
  11. 2024Imitation Learning in Discounted Linear MDPs without exploration assumptionscited by 15
  12. 2022Structural Estimation of Markov Decision Processes in High-Dimensional State Space with Finite-Time Guaranteescited by 12
21 more
  1. 2024A Comprehensive Survey on Inverse Constrained Reinforcement Learning: Definitions, Progress and Challengescited by 10
  2. 2022Robust Learning from Observation with Model Misspecificationcited by 9
  3. 2023Concept Alignment as a Prerequisite for Value Alignmentcited by 5
  4. 2025IL-SOAR : Imitation Learning with Soft Optimistic Actor cRiticcited by 5
  5. 2021Robust Entropy-regularized Markov Decision Processescited by 4
  6. 2024Quantifying the Sensitivity of Inverse Reinforcement Learning to Misspecificationcited by 4
  7. 2024OMPO: A Unified Framework for RL under Policy and Dynamics Shiftscited by 4
  8. 2025Rectified Robust Policy Optimization for Model-Uncertain Constrained Reinforcement Learning without Strong Dualitycited by 4
  9. 2024Enhancing Inverse Reinforcement Learning through Encoding Dynamic Information in Reward Shapingcited by 3
  10. 2024Rethinking Inverse Reinforcement Learning: from Data Alignment to Task Alignmentcited by 3
  11. 2024Learning Causally Invariant Reward Functions from Diverse Demonstrationscited by 2
  12. 2026Direct Preference Optimization with Rating Information: Practical Algorithms and Provable Gainscited by 2
  13. 2025Inversely Learning Transferable Rewards via Abstracted Statescited by 1
  14. 2025Statistical Scenario Modelling and Lookalike Distributions for Multi-Variate AI Riskcited by 1
  15. 2025Generalizing Behavior via Inverse Reinforcement Learning with Closed-Form Reward Centroidscited by 1
  16. 2025Balance Equation-based Distributionally Robust Offline Imitation Learningcited by 1
  17. 2022User-Oriented Robust Reinforcement Learningcited by 0
  18. 2023A Novel Variational Lower Bound for Inverse Reinforcement Learningcited by 0
  19. 2025Robustness in the Face of Partial Identifiability in Reward Learningcited by 0
  20. 2025Distributional Inverse Reinforcement Learningcited by 0
  21. 2026Unifying Value Alignment and Assignment in Cross-Domain Offline Reinforcement Learning with Heterogeneous Datasetscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2019Generalized Maximum Causal Entropy for Inverse Reinforcement Learning
  2. 2022A Primer on Maximum Causal Entropy Inverse Reinforcement Learning
  3. 2020Revisiting Maximum Entropy Inverse Reinforcement Learning: New Perspectives and Algorithms
  4. 2026Fast Rates for Inverse Reinforcement Learning
  5. 2021Inverse Reinforcement Learning in a Continuous State Space with Formal Guarantees
  6. 2018Multi-task Maximum Entropy Inverse Reinforcement Learning
  7. 2023A Bayesian Approach to Robust Inverse Reinforcement Learning
  8. 2022Weighted Maximum Entropy Inverse Reinforcement Learning