mlsanityData to 7 Oct 2026

BC-IRL: Learning Generalizable Reward Functions from Demonstrations

arXiv 2303.16194 ↗First posted 28 Mar 2023cs.LGcited by 7

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

How well do reward functions learned with inverse reinforcement learning (IRL) generalize? We illustrate that state-of-the-art IRL algorithms, which maximize a maximum-entropy objective, learn rewards that overfit to the demonstrations. Such rewards struggle to provide meaningful rewards for states not covered by the demonstrations, a major detriment when using the reward to learn policies in new situations. We introduce BC-IRL a new inverse reinforcement learning method that learns reward functions that generalize better when compared to maximum-entropy IRL approaches. In contrast to the MaxEnt framework, which learns to maximize rewards around demonstrations, BC-IRL updates reward parameters such that the policy trained with the new reward matches the expert demonstrations better. We show that BC-IRL learns rewards that generalize better on an illustrative simple task and two continuous robotic control tasks, achieving over twice the success rate of baselines in challenging generalization settings.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 31 of its 45 references that are in the map. First posted on arXiv 28 Mar 2023.

Builds on

31 of 45 references.

Closest ancestors · 5

Same direction, newest first

  1. 2021Meta-Adversarial Inverse Reinforcement Learning for Decision-making Tasks23 Mar
  2. 2020f-IRL: Inverse Reinforcement Learning via State Marginal Matching9 Nov
  3. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variables20 Sep
  4. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learning31 May
  5. 2018Multi-task Maximum Entropy Inverse Reinforcement Learning22 May

Other references · 26

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2015Continuous control with deep reinforcement learningcited by 3,279
  6. 2016Generative Adversarial Imitation Learningcited by 1,576
  7. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  8. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  9. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  10. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  11. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  12. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
14 more
  1. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  2. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  3. 2020Primal Wasserstein Imitation Learningcited by 120
  4. 2020CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learningcited by 109
  5. 2019Generalized Inner Loop Meta-Learningcited by 92
  6. 2021What Matters for Adversarial Imitation Learning?cited by 70
  7. 2019Meta-Learning via Learned Losscited by 60
  8. 2020Model-Based Inverse Reinforcement Learning from Visual Demonstrationscited by 52
  9. 2019Policies Modulating Trajectory Generatorscited by 47
  10. 2019Task-Relevant Adversarial Imitation Learningcited by 45
  11. 2019Positive-Unlabeled Reward Learningcited by 36
  12. 2020Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimizationcited by 22
  13. 2021Imitation by Predicting Observationscited by 13
  14. 2020Combating False Negatives in Adversarial Imitation Learningcited by 4

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 2; 2025: 2.

7 papers.

  1. 2023Coherent Soft Imitation Learningcited by 21
  2. 2025Enhancing Rating-Based Reinforcement Learning to Effectively Leverage Feedback from Large Vision-Language Modelscited by 15
  3. 2023Rating-based Reinforcement Learningcited by 14
  4. 2023FoMo Rewards: Can we cast foundation models as reward functions?cited by 4
  5. 2025Trust Region Reward Optimization and Proximal Inverse Reward Optimization Algorithmcited by 1
  6. 2024Rethinking Adversarial Inverse Reinforcement Learning: Policy Imitation, Transferable Reward Recovery and Algebraic Equilibrium Proofcited by 0
  7. 2024On Reward Transferability in Adversarial Inverse Reinforcement Learning: Insights from Random Matrix Theorycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Generalizing Behavior via Inverse Reinforcement Learning with Closed-Form Reward Centroids
  2. 2020Inverse Reinforcement Learning via Matching of Optimality Profiles
  3. 2020Revisiting Maximum Entropy Inverse Reinforcement Learning: New Perspectives and Algorithms
  4. 2020Efficient Exploration of Reward Functions in Inverse Reinforcement Learning via Bayesian Optimization
  5. 2026Generalize and Guide: Decomposing Rewards for Few-Shot Inverse Reinforcement Learning
  6. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
  7. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress
  8. 2025Reward Compatibility: A Framework for Inverse RL