mlsanityData to 7 Oct 2026

Environment Design for Inverse Reinforcement Learning

arXiv 2210.14972 ↗First posted 26 Oct 2022cs.LG cs.AIcited by 4

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Learning a reward function from demonstrations suffers from low sample-efficiency. Even with abundant data, current inverse reinforcement learning methods that focus on learning from a single environment can fail to handle slight changes in the environment dynamics. We tackle these challenges through adaptive environment design. In our framework, the learner repeatedly interacts with the expert, with the former selecting environments to identify the reward function as quickly as possible from the expert's demonstrations in said environments. This results in improvements in both sample-efficiency and robustness, as we show experimentally, for both exact and approximate inference.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 29 of its 56 references that are in the map. First posted on arXiv 26 Oct 2022.

Builds on

29 of 56 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022Identifiability and generalizability from multiple experts in Inverse Reinforcement Learning22 Sep
  2. 2022Active Exploration for Inverse Reinforcement Learning18 Jul
  3. 2021Identifiability in inverse reinforcement learning7 Jun
  4. 2021Scalable Bayesian Inverse Reinforcement Learning12 Feb
  5. 2019Risk-Aware Active Inverse Reinforcement Learning8 Jan
  6. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress18 Jun

Other references · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2016Generative Adversarial Imitation Learningcited by 1,576
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  5. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  6. 2019Solving Rubik's Cube with a Robot Handcited by 699
  7. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  8. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  9. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 350
  10. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 260
  11. 2014Learning to Optimize via Information-Directed Samplingcited by 168
  12. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
11 more
  1. 2020Reward-rational (implicit) choice: A unifying formalism for reward learningcited by 119
  2. 2017Reinforcement Learning under Model Mismatchcited by 52
  3. 2022Environment Generation for Zero-Shot Compositional Reinforcement Learningcited by 32
  4. 2020The MAGICAL Benchmark for Robust Imitationcited by 30
  5. 2017Repeated Inverse Reinforcement Learningcited by 29
  6. 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
  7. 2021Information Directed Reward Learning for Reinforcement Learningcited by 24
  8. 2018Configurable Markov Decision Processescited by 9
  9. 2021Interactive Inverse Reinforcement Learning for Cooperative Gamescited by 7
  10. 2019Inverse Reinforcement Learning with Multiple Ranked Expertscited by 5
  11. 2021Assisted Robust Reward Designcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 2; 2026: 1.

4 papers.

  1. 2023Eliciting Risk Aversion with Inverse Reinforcement Learning via Interactive Questioningcited by 2
  2. 2025Inversely Learning Transferable Rewards via Abstracted Statescited by 1
  3. 2025PAC Apprenticeship Learning with Bayesian Active Inverse Reinforcement Learningcited by 1
  4. 2026ConTraIRL: Factorized Contrastive Abstractions for Transferable IRLcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Inverse Reinforcement Learning from a Gradient-based Learner
  2. 2026Multi-Modal, Multi-Environment Machine Teaching for Robust Reward Learning
  3. 2023When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learning
  4. 2019Option Compatible Reward Inverse Reinforcement Learning
  5. 2021Sample Efficient Imitation Learning via Reward Function Trained in Advance
  6. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learning
  7. 2023A Bayesian Approach to Robust Inverse Reinforcement Learning
  8. 2021OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matching