mlsanityData to 14 Oct 2026

Teaching Inverse Reinforcement Learners via Features and Demonstrations

arXiv 1810.08926 ↗First posted 21 Oct 2018cs.LG stat.MLcited by 19

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Learning near-optimal behaviour from an expert's demonstrations typically relies on the assumption that the learner knows the features that the true reward function depends on. In this paper, we study the problem of learning from demonstrations in the setting where this is not the case, i.e., where there is a mismatch between the worldviews of the learner and the expert. We introduce a natural quantity, the teaching risk, which measures the potential suboptimality of policies that look optimal to the learner in this setting. We show that bounds on the teaching risk guarantee that the learner is able to find a near-optimal policy using standard algorithms based on inverse reinforcement learning. Based on these findings, we suggest a teaching scheme in which the expert can decrease the teaching risk by updating the learner's worldview, and thus ultimately enable her to find a near-optimal policy.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. First posted on arXiv 21 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 2; 2020: 1; 2021: 8; 2022: 3; 2023: 3; 2024: 0; 2025: 1; 2026: 1.

19 papers.

Sort:

By area:

  1. 2020Robust Inverse Reinforcement Learning under Transition Dynamics Mismatchcited by 33
  2. 2021Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environmentscited by 26
  3. 2021Reinforcement Learning for Education: Opportunities and Challengescited by 25
  4. 2019Interactive Teaching Algorithms for Inverse Reinforcement Learningcited by 24
  5. 2019Learner-aware Teaching: Inverse Reinforcement Learning with Preferences and Constraintscited by 18
  6. 2022Inducing Structure in Reward Learning by Learning Featurescited by 16
  7. 2021Reinforcement Learning Under Algorithmic Triagecited by 15
  8. 2021Teaching an Active Learner with Contrastive Examplescited by 12
  9. 2021Fair Clustering Using Antidote Datacited by 10
  10. 2021Iterative Teaching by Label Synthesiscited by 10
  11. 2021Curriculum Design for Teaching via Demonstrations: Theory and Applicationscited by 9
  12. 2022Iterative Teaching by Data Hallucinationcited by 8
7 more
  1. 2021Iterative Teacher-Aware Learningcited by 7
  2. 2022Learning from Physical Human Feedback: An Object-Centric One-Shot Adaptation Methodcited by 2
  3. 2023Active Third-Person Imitation Learningcited by 1
  4. 2026Multi-Modal, Multi-Environment Machine Teaching for Robust Reward Learningcited by 1
  5. 2023Optimally Teaching a Linear Behavior Cloning Agentcited by 0
  6. 2023Model Selection for Inverse Reinforcement Learning via Structural Risk Minimizationcited by 0
  7. 2025Towards User-Centred Design of AI-Assisted Decision-Making in Law Enforcementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2018Machine Teaching for Inverse Reinforcement Learning: Algorithms and Applications
  2. 2020Inverse Reinforcement Learning from a Gradient-based Learner
  3. 2023Interactively Teaching an Inverse Reinforcement Learner with Limited Feedback
  4. 2020Inverse Reinforcement Learning via Matching of Optimality Profiles
  5. 2025Generalizing Behavior via Inverse Reinforcement Learning with Closed-Form Reward Centroids
  6. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
  7. 2020Learning from Suboptimal Demonstration via Self-Supervised Reward Regression
  8. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learning