mlsanityData to 13 Oct 2026

Meta-Inverse Reinforcement Learning with Probabilistic Context Variables

arXiv 1909.09314 ↗First posted 20 Sep 2019cs.LG stat.MLcited by 50

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Providing a suitable reward function to reinforcement learning can be difficult in many real world applications. While inverse reinforcement learning (IRL) holds promise for automatically learning reward functions from demonstrations, several major challenges remain. First, existing IRL methods learn reward functions from scratch, requiring large numbers of demonstrations to correctly infer the reward for each task the agent may need to perform. Second, existing methods typically assume homogeneous demonstrations for a single behavior or task, while in practice, it might be easier to collect datasets of heterogeneous but related behaviors. To this end, we propose a deep latent variable model that is capable of learning rewards from demonstrations of distinct but related tasks in an unsupervised way. Critically, our model can infer rewards for new, structurally-similar tasks from a single demonstration. Our experiments on multiple continuous control tasks demonstrate the effectiveness of our approach compared to state-of-the-art imitation and inverse reinforcement learning methods.

Key facts. Cited by 50 papers in the map (data through 13 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 26 of its 42 references that are in the map. First posted on arXiv 20 Sep 2019.

Builds on

26 of 42 references.

Closest ancestors · 2

Same direction, newest first

  1. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learning31 May
  2. 2018Multi-task Maximum Entropy Inverse Reinforcement Learning22 May

Other references · 24

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,870
  2. 2013Auto-Encoding Variational Bayescited by 4,024
  3. 2015Trust Region Policy Optimizationcited by 2,513
  4. 2016Concrete Problems in AI Safetycited by 1,614
  5. 2016Generative Adversarial Imitation Learningcited by 1,577
  6. 2016Learning to learn by gradient descent by gradient descentcited by 617
  7. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 494
  8. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 486
  9. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 477
  10. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  11. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 373
  12. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
12 more
  1. 2017Meta Networkscited by 269
  2. 2017One-Shot Imitation Learningcited by 232
  3. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  4. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  5. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 146
  6. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flowcited by 103
  7. 2017Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Netscited by 64
  8. 2018Meta Reinforcement Learning with Latent Variable Gaussian Processescited by 44
  9. 2017Learning to Optimize Neural Netscited by 41
  10. 2018Directed-Info GAIL: Learning Hierarchical Policies from Unsegmented Demonstrations using Directed Informationcited by 41
  11. 2018One-Shot Hierarchical Imitation Learning of Compound Visuomotor Taskscited by 32
  12. 2017Burn-In Demonstrations for Multi-Modal Imitation Learningcited by 9

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 15; 2022: 6; 2023: 8; 2024: 7; 2025: 8; 2026: 6.

50 papers.

Sort:

By area:

  1. 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 191
  2. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
  3. 2022Why So Pessimistic? Estimating Uncertainties for Offline RL through Ensembles, and Why Their Independence Matterscited by 74
  4. 2021Procedure Planning in Instructional Videos via Contextual Modeling and Model-based Policy Learningcited by 53
  5. 2021Baby Intuitions Benchmark (BIB): Discerning the goals, preferences, and actions of otherscited by 39
  6. 2021Hard Choices in Artificial Intelligencecited by 26
  7. 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
  8. 2021MORAL: Aligning AI with Human Norms through Multi-Objective Reinforced Active Learningcited by 24
  9. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  10. 2023Learning Safety Constraints from Demonstrations with Unknown Rewardscited by 19
  11. 2021OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matchingcited by 15
  12. 2023Multi-task Hierarchical Adversarial Inverse Reinforcement Learningcited by 15
38 more
  1. 2023Learning Shared Safety Constraints from Multi-task Demonstrationscited by 15
  2. 2023SIRL: Similarity-based Implicit Representation Learningcited by 13
  3. 2023A Survey of Demonstration Learningcited by 12
  4. 2023Aligning Robot and Human Representationscited by 11
  5. 2021Adversarial Inverse Reinforcement Learning for Mean Field Gamescited by 8
  6. 2022Transfering Hierarchical Structure with Dual Meta Imitation Learningcited by 7
  7. 2023BC-IRL: Learning Generalizable Reward Functions from Demonstrationscited by 7
  8. 2025GoIRL: Graph-Oriented Inverse Reinforcement Learning for Multimodal Trajectory Predictioncited by 7
  9. 2021Grounded Relational Inference: Domain Knowledge Driven Explainable Autonomous Drivingcited by 6
  10. 2021Meta-Adversarial Inverse Reinforcement Learning for Decision-making Taskscited by 6
  11. 2021Learning without Knowing: Unobserved Context in Continuous Transfer Reinforcement Learningcited by 4
  12. 2022Basis for Intentions: Efficient Inverse Reinforcement Learning using Past Experiencecited by 4
  13. 2024Diversifying Policy Behaviors with Extrinsic Behavioral Curiositycited by 4
  14. 2024Identifying Selections for Unsupervised Subtask Discoverycited by 4
  15. 2021Off-Dynamics Inverse Reinforcement Learning from Hetero-Domaincited by 3
  16. 2025Meta-Inverse Reinforcement Learning for Mean Field Games via Probabilistic Context Variablescited by 3
  17. 2021LiMIIRL: Lightweight Multiple-Intent Inverse Reinforcement Learningcited by 2
  18. 2024Sequential Decision Making with Expert Demonstrations under Unobserved Heterogeneitycited by 1
  19. 2025Inversely Learning Transferable Rewards via Abstracted Statescited by 1
  20. 2025Trust Region Reward Optimization and Proximal Inverse Reward Optimization Algorithmcited by 1
  21. 2025Consistent Zero-Shot Imitation with Contrastive Goal Inferencecited by 1
  22. 2021Quickest change detection with unknown parameters: Constant complexity and near optimalitycited by 0
  23. 2021SS-MAIL: Self-Supervised Multi-Agent Imitation Learningcited by 0
  24. 2022Context-Hierarchy Inverse Reinforcement Learningcited by 0
  25. 2022Learning Latent Traits for Simulated Cooperative Driving Taskscited by 0
  26. 2023Kernel Density Bayesian Inverse Reinforcement Learningcited by 0
  27. 2024Multi Task Inverse Reinforcement Learning for Common Sense Rewardcited by 0
  28. 2024Semi-Supervised One-Shot Imitation Learningcited by 0
  29. 2024Inverse Reinforcement Learning with Multiple Planning Horizonscited by 0
  30. 2025Supervised Reward Inferencecited by 0
  31. 2025Recursive Deep Inverse Reinforcement Learningcited by 0
  32. 2025Multi-Mode Process Control Using Multi-Task Inverse Reinforcement Learningcited by 0
  33. 2026ConTraIRL: Factorized Contrastive Abstractions for Transferable IRLcited by 0
  34. 2026Do as the Romans Do: Learning Universal Behaviors from Heterogeneous Agentscited by 0
  35. 2026Multi-Modal, Multi-Environment Machine Teaching for Robust Reward Learningcited by 0
  36. 2026Generalize and Guide: Decomposing Rewards for Few-Shot Inverse Reinforcement Learningcited by 0
  37. 2026Does Latent Context Help? A Controlled Evaluation of Inverse Reinforcement Learning in Arctic Shippingcited by 0
  38. 2026Scalable Multi-Task Inverse Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Learning Causally Invariant Reward Functions from Diverse Demonstrations
  2. 2020Efficient Exploration of Reward Functions in Inverse Reinforcement Learning via Bayesian Optimization
  3. 2024Automated Feature Selection for Inverse Reinforcement Learning
  4. 2020Inverse Reinforcement Learning via Matching of Optimality Profiles
  5. 2020Active Task-Inference-Guided Deep Inverse Reinforcement Learning
  6. 2021Versatile Inverse Reinforcement Learning via Cumulative Rewards
  7. 2024Rethinking Inverse Reinforcement Learning: from Data Alignment to Task Alignment
  8. 2019Option Compatible Reward Inverse Reinforcement Learning