mlsanityData to 7 Oct 2026

Learning a Prior over Intent via Meta-Inverse Reinforcement Learning

arXiv 1805.12573 ↗First posted 31 May 2018cs.LG stat.MLcited by 36

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

A significant challenge for the practical application of reinforcement learning in the real world is the need to specify an oracle reward function that correctly defines a task. Inverse reinforcement learning (IRL) seeks to avoid this challenge by instead inferring a reward function from expert behavior. While appealing, it can be impractically expensive to collect datasets of demonstrations that cover the variation common in the real world (e.g. opening any type of door). Thus in practice, IRL must commonly be performed with only a limited set of demonstrations where it can be exceedingly difficult to unambiguously recover a reward function. In this work, we exploit the insight that demonstrations from other tasks can be used to constrain the set of possible reward functions by learning a "prior" that is specifically optimized for the ability to infer expressive reward functions from limited numbers of demonstrations. We demonstrate that our method can efficiently recover rewards from images for novel tasks and provide intuition as to how our approach is analogous to learning a prior.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 31 of its 65 references that are in the map. First posted on arXiv 31 May 2018.

Builds on

31 of 65 references.

References · 31

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  5. 2016Matching Networks for One Shot Learningcited by 2,246
  6. 2016Concrete Problems in AI Safetycited by 1,582
  7. 2016Generative Adversarial Imitation Learningcited by 1,576
  8. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  9. 2016Learning to learn by gradient descent by gradient descentcited by 616
  10. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  11. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  12. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
19 more
  1. 2016Learning to reinforcement learncited by 369
  2. 2017One-Shot Imitation Learningcited by 232
  3. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  4. 2014Recurrent Neural Network Regularizationcited by 189
  5. 2018Recasting Gradient-Based Meta-Learning as Hierarchical Bayescited by 180
  6. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
  7. 2017Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Netscited by 64
  8. 2016One-Shot Generalization in Deep Generative Modelscited by 44
  9. 2017Learning to Optimize Neural Netscited by 41
  10. 2016Watch This: Scalable Cost-Function Learning for Path Planning in Urban Environmentscited by 33
  11. 2017Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributionscited by 24
  12. 2015Early Stopping is Nonparametric Variational Inferencecited by 23
  13. 2017A Simple Neural Attentive Meta-Learnercited by 22
  14. 2017Attentive Recurrent Comparatorscited by 13
  15. 2016TorchCraft: a Library for Machine Learning Research on Real-Time Strategy Gamescited by 12
  16. 2017InfoGAIL: Interpretable Imitation Learning from Visual Demonstrationscited by 11
  17. 2017Burn-In Demonstrations for Multi-Modal Imitation Learningcited by 9
  18. 2016Incorporating Human Domain Knowledge into Large Scale Cost Function Learningcited by 5
  19. 2017Meta Inverse Reinforcement Learning via Maximum Reward Sharing for Human Motion Analysiscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 2; 2020: 1; 2021: 8; 2022: 5; 2023: 10; 2024: 3; 2025: 5; 2026: 2.

36 papers.

Sort:

By area:

  1. 2023Getting aligned on representational alignmentcited by 142
  2. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
  3. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variablescited by 50
  4. 2021Baby Intuitions Benchmark (BIB): Discerning the goals, preferences, and actions of otherscited by 39
  5. 2020Bayesian Robust Optimization for Imitation Learningcited by 28
  6. 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
  7. 2021MORAL: Aligning AI with Human Norms through Multi-Objective Reinforced Active Learningcited by 24
  8. 2023NOPA: Neurally-guided Online Probabilistic Assistance for Building Socially Intelligent Home Assistantscited by 23
  9. 2023Learning Safety Constraints from Demonstrations with Unknown Rewardscited by 19
  10. 2023Theory of Mind as Intrinsic Motivation for Multi-Agent Reinforcement Learningcited by 19
  11. 2021OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matchingcited by 15
  12. 2023Learning Shared Safety Constraints from Multi-task Demonstrationscited by 15
24 more
  1. 2023SIRL: Similarity-based Implicit Representation Learningcited by 13
  2. 2019Inverse Reinforcement Learning in Contextual MDPscited by 12
  3. 2021Policy Gradient Bayesian Robust Optimization for Imitation Learningcited by 11
  4. 2023Aligning Robot and Human Representationscited by 11
  5. 2024The Role of Predictive Uncertainty and Diversity in Embodied AI and Robot Learningcited by 11
  6. 2023Designing Fiduciary Artificial Intelligencecited by 8
  7. 2021Combining Reward Information from Multiple Sourcescited by 7
  8. 2022Transfering Hierarchical Structure with Dual Meta Imitation Learningcited by 7
  9. 2023BC-IRL: Learning Generalizable Reward Functions from Demonstrationscited by 7
  10. 2024In-Trajectory Inverse Reinforcement Learning: Learn Incrementally Before An Ongoing Trajectory Terminatescited by 7
  11. 2021Meta-Adversarial Inverse Reinforcement Learning for Decision-making Taskscited by 6
  12. 2022Basis for Intentions: Efficient Inverse Reinforcement Learning using Past Experiencecited by 4
  13. 2022Discovering Generalizable Spatial Goal Representations via Graph-based Active Reward Learningcited by 3
  14. 2025Meta-Inverse Reinforcement Learning for Mean Field Games via Probabilistic Context Variablescited by 3
  15. 2021SS-MAIL: Self-Supervised Multi-Agent Imitation Learningcited by 0
  16. 2022Aligning Robot Representations with Humanscited by 0
  17. 2023Kernel Density Bayesian Inverse Reinforcement Learningcited by 0
  18. 2024Multi Task Inverse Reinforcement Learning for Common Sense Rewardcited by 0
  19. 2025Enhancing Online Reinforcement Learning with Meta-Learned Objective from Offline Datacited by 0
  20. 2025Supervised Reward Inferencecited by 0
  21. 2025Recursive Deep Inverse Reinforcement Learningcited by 0
  22. 2025QuickLAP: Quick Language-Action Preference Learning for Semi-Autonomous Agentscited by 0
  23. 2026Generalize and Guide: Decomposing Rewards for Few-Shot Inverse Reinforcement Learningcited by 0
  24. 2026Does Latent Context Help? A Controlled Evaluation of Inverse Reinforcement Learning in Arctic Shippingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2019Option Compatible Reward Inverse Reinforcement Learning
  2. 2020Efficient Exploration of Reward Functions in Inverse Reinforcement Learning via Bayesian Optimization
  3. 2020Inverse Reinforcement Learning via Matching of Optimality Profiles
  4. 2021Versatile Inverse Reinforcement Learning via Cumulative Rewards
  5. 2020Inverse Reinforcement Learning from a Gradient-based Learner
  6. 2020Active Task-Inference-Guided Deep Inverse Reinforcement Learning
  7. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
  8. 2023Distance-rank Aware Sequential Reward Learning for Inverse Reinforcement Learning with Sub-optimal Demonstrations