mlsanityData to 14 Oct 2026

Machine Teaching for Inverse Reinforcement Learning: Algorithms and Applications

arXiv 1805.07687 ↗First posted 20 May 2018cs.LG stat.MLcited by 28

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Inverse reinforcement learning (IRL) infers a reward function from demonstrations, allowing for policy improvement and generalization. However, despite much recent interest in IRL, little work has been done to understand the minimum set of demonstrations needed to teach a specific sequential decision-making task. We formalize the problem of finding maximally informative demonstrations for IRL as a machine teaching problem where the goal is to find the minimum number of demonstrations needed to specify the reward equivalence class of the demonstrator. We extend previous work on algorithmic teaching for sequential decision-making tasks by showing a reduction to the set cover problem which enables an efficient approximation algorithm for determining the set of maximally-informative demonstrations. We apply our proposed machine teaching algorithm to two novel applications: providing a lower bound on the number of queries needed to learn a policy using active IRL and developing a novel IRL algorithm that can learn more efficiently from informative demonstrations than a standard IRL approach.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 11 of its 68 references that are in the map. First posted on arXiv 20 May 2018.

Builds on

11 of 68 references.

References · 11

  1. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  2. 2016Successor Features for Transfer in Reinforcement Learningcited by 340
  3. 2016Cooperative Inverse Reinforcement Learningcited by 300
  4. 2018An Overview of Machine Teachingcited by 63
  5. 2017Enabling Robots to Communicate their Objectivescited by 35
  6. 2014Near-Optimally Teaching the Crowd to Classifycited by 32
  7. 2015The Teaching Dimension of Linear Learnerscited by 19
  8. 2013Machine Teaching for Bayesian Learners in the Exponential Familycited by 18
  9. 2017Efficient Probabilistic Performance Bounds for Inverse Reinforcement Learningcited by 17
  10. 2017Preference-based Teachingcited by 11
  11. 2014Exploiting Model Equivalences for Solving Interactive Dynamic Influence Diagramscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 4; 2020: 3; 2021: 9; 2022: 4; 2023: 4; 2024: 0; 2025: 2; 2026: 2.

28 papers.

Sort:

By area:

  1. 2020Human-in-the-Loop Imitation Learning using Remote Teleoperationcited by 112
  2. 2019Policy Poisoning in Batch Reinforcement Learning and Controlcited by 70
  3. 2021Self-Supervised Online Reward Shaping in Sparse-Reward Environmentscited by 37
  4. 2021Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environmentscited by 26
  5. 2019Interactive Teaching Algorithms for Inverse Reinforcement Learningcited by 24
  6. 2019Learner-aware Teaching: Inverse Reinforcement Learning with Preferences and Constraintscited by 18
  7. 2020Efficient Exploration of Reward Functions in Inverse Reinforcement Learning via Bayesian Optimizationcited by 16
  8. 2021Reinforcement Learning Under Algorithmic Triagecited by 15
  9. 2021Fair Clustering Using Antidote Datacited by 10
  10. 2021Curriculum Design for Teaching via Demonstrations: Theory and Applicationscited by 9
  11. 2021Interactive Inverse Reinforcement Learning for Cooperative Gamescited by 7
  12. 2019Better-than-Demonstrator Imitation Learning via Automatically-Ranked Demonstrationscited by 6
16 more
  1. 2021Learning Data Teaching Strategies Via Knowledge Tracingcited by 6
  2. 2022Reasoning about Counterfactuals to Improve Human Inverse Reinforcement Learningcited by 6
  3. 2023Non-Clashing Teaching Maps for Balls in Graphscited by 6
  4. 2025The Computational Complexity of Positive Non-Clashing Teaching in Graphscited by 3
  5. 2021Dynamically Switching Human Prediction Models for Efficient Planningcited by 2
  6. 2022Reinforcement Teachingcited by 1
  7. 2022Autonomous Assessment of Demonstration Sufficiency via Bayesian Inverse Reinforcement Learningcited by 1
  8. 2023Utility-based Adaptive Teaching Strategies using Bayesian Theory of Mindcited by 1
  9. 2025Inverse Reinforcement Learning for Minimum-Exposure Paths in Spatiotemporally Varying Scalar Fieldscited by 1
  10. 2026Non-Clashing Teaching in Graphs: Algorithms, Complexity, and Boundscited by 1
  11. 2026Multi-Modal, Multi-Environment Machine Teaching for Robust Reward Learningcited by 1
  12. 2020Interaction-limited Inverse Reinforcement Learningcited by 0
  13. 2021Address Behaviour Vulnerabilities in the Next Generation of Autonomous Robotscited by 0
  14. 2022Know Thy Student: Interactive Learning with Gaussian Processescited by 0
  15. 2023Interactively Teaching an Inverse Reinforcement Learner with Limited Feedbackcited by 0
  16. 2023Optimally Teaching a Linear Behavior Cloning Agentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2018Teaching Inverse Reinforcement Learners via Features and Demonstrations
  2. 2023BC-IRL: Learning Generalizable Reward Functions from Demonstrations
  3. 2020Inverse Reinforcement Learning via Matching of Optimality Profiles
  4. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
  5. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variables
  6. 2020Inverse Reinforcement Learning from a Gradient-based Learner
  7. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress
  8. 2024Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review