mlsanityData to 14 Oct 2026

Off-Dynamics Inverse Reinforcement Learning from Hetero-Domain

arXiv 2110.11443 ↗First posted 21 Oct 2021cs.LGcited by 3

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

We propose an approach for inverse reinforcement learning from hetero-domain which learns a reward function in the simulator, drawing on the demonstrations from the real world. The intuition behind the method is that the reward function should not only be oriented to imitate the experts, but should encourage actions adjusted for the dynamics difference between the simulator and the real world. To achieve this, the widely used GAN-inspired IRL method is adopted, and its discriminator, recognizing policy-generating trajectories, is modified with the quantification of dynamics difference. The training process of the discriminator can yield the transferable reward function suitable for simulator dynamics, which can be guaranteed by derivation. Effectively, our method assigns higher rewards for demonstration trajectories which do not exploit discrepancies between the two domains. With extensive experiments on continuous control tasks, our method shows its effectiveness and demonstrates its scalability to high-dimensional tasks.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 17 of its 29 references that are in the map. First posted on arXiv 21 Oct 2021.

Builds on

17 of 29 references.

Closest ancestors · 2

Same direction, newest first

  1. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variables20 Sep
  2. 2018Adversarial Imitation via Variational Inverse Reinforcement Learning17 Sep

Other references · 15

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2016OpenAI Gymcited by 1,736
  4. 2016Generative Adversarial Imitation Learningcited by 1,597
  5. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  6. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  7. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 406
  8. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 289
  9. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 168
  10. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 148
  11. 2020Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifierscited by 97
  12. 2019State Alignment-based Imitation Learningcited by 73
3 more
  1. 2016Control of Memory, Active Perception, and Action in Minecraftcited by 61
  2. 2021Cross-domain Imitation from Observationscited by 42
  3. 2020State-only Imitation with Transition Dynamics Mismatchcited by 37

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1.

3 papers.

  1. 2023Behavior Proximal Policy Optimizationcited by 43
  2. 2023Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learningcited by 27
  3. 2024Off-dynamics Conditional Diffusion Plannerscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Off-Policy Adversarial Inverse Reinforcement Learning
  2. 2025OffSim: Offline Simulator for Model-based Offline Inverse Reinforcement Learning
  3. 2024Automated Feature Selection for Inverse Reinforcement Learning
  4. 2021Versatile Inverse Reinforcement Learning via Cumulative Rewards
  5. 2023CLARE: Conservative Model-Based Reward Learning for Offline Inverse Reinforcement Learning
  6. 2021OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matching
  7. 2022Transferable Reward Learning by Dynamics-Agnostic Discriminator Ensemble
  8. 2025Offline Reinforcement Learning via Inverse Optimization