mlsanityData to 14 Oct 2026

Interactive Teaching Algorithms for Inverse Reinforcement Learning

arXiv 1905.11867 ↗First posted 28 May 2019cs.LG cs.AI stat.MLcited by 24

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

We study the problem of inverse reinforcement learning (IRL) with the added twist that the learner is assisted by a helpful teacher. More formally, we tackle the following algorithmic question: How could a teacher provide an informative sequence of demonstrations to an IRL learner to speed up the learning process? We present an interactive teaching framework where a teacher adaptively chooses the next demonstration based on learner's current policy. In particular, we design teaching algorithms for two concrete settings: an omniscient setting where a teacher has full knowledge about the learner's dynamics and a blackbox setting where the teacher has minimal knowledge. Then, we study a sequential variant of the popular MCE-IRL learner and prove convergence guarantees of our teaching algorithm in the omniscient setting. Extensive experiments with a car driving simulator environment show that the learning progress can be speeded up drastically as compared to an uninformative teacher.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 17 of its 60 references that are in the map. First posted on arXiv 28 May 2019.

Builds on

17 of 60 references.

Closest ancestors · 2

Same direction, newest first

  1. 2018Teaching Inverse Reinforcement Learners via Features and Demonstrations21 Oct
  2. 2018Machine Teaching for Inverse Reinforcement Learning: Algorithms and Applications20 May

Other references · 15

  1. 2016Generative Adversarial Imitation Learningcited by 1,597
  2. 2018An Algorithmic Perspective on Imitation Learningcited by 459
  3. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 406
  4. 2016Cooperative Inverse Reinforcement Learningcited by 300
  5. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 176
  6. 2018An Overview of Machine Teachingcited by 63
  7. 2017Iterative Machine Teachingcited by 41
  8. 2016First-Person Activity Forecasting with Online Inverse Reinforcement Learningcited by 40
  9. 2014Near-Optimally Teaching the Crowd to Classifycited by 32
  10. 2017Repeated Inverse Reinforcement Learningcited by 29
  11. 2018Dual Policy Iterationcited by 22
  12. 2017Towards Black-box Iterative Machine Teachingcited by 20
3 more
  1. 2018Teaching Multiple Concepts to a Forgetful Learnercited by 17
  2. 2018Understanding the Role of Adaptivity in Machine Teaching: The Case of Version Space Learnerscited by 16
  3. 2018Iterative Classroom Teachingcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 2; 2020: 2; 2021: 8; 2022: 3; 2023: 6; 2024: 0; 2025: 1; 2026: 2.

24 papers.

Sort:

By area:

  1. 2019Policy Poisoning in Batch Reinforcement Learning and Controlcited by 70
  2. 2020Robust Inverse Reinforcement Learning under Transition Dynamics Mismatchcited by 33
  3. 2021Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environmentscited by 26
  4. 2021Reinforcement Learning for Education: Opportunities and Challengescited by 25
  5. 2023Proximal Curriculum for Reinforcement Learning Agentscited by 20
  6. 2019Learner-aware Teaching: Inverse Reinforcement Learning with Preferences and Constraintscited by 18
  7. 2021Reinforcement Learning Under Algorithmic Triagecited by 15
  8. 2021Teaching an Active Learner with Contrastive Examplescited by 12
  9. 2021Fair Clustering Using Antidote Datacited by 10
  10. 2021Iterative Teaching by Label Synthesiscited by 10
  11. 2021Curriculum Design for Teaching via Demonstrations: Theory and Applicationscited by 9
  12. 2023Nonparametric Iterative Machine Teachingcited by 9
12 more
  1. 2022Iterative Teaching by Data Hallucinationcited by 8
  2. 2021Interactive Inverse Reinforcement Learning for Cooperative Gamescited by 7
  3. 2023Nonparametric Teaching for Multiple Learnerscited by 7
  4. 2025LLM-Mediated Guidance of MARL Systemscited by 3
  5. 2022Autonomous Assessment of Demonstration Sufficiency via Bayesian Inverse Reinforcement Learningcited by 1
  6. 2023Active Third-Person Imitation Learningcited by 1
  7. 2026Multi-Modal, Multi-Environment Machine Teaching for Robust Reward Learningcited by 1
  8. 2020Interaction-limited Inverse Reinforcement Learningcited by 0
  9. 2022One-shot Machine Teaching: Cost Very Few Examples to Converge Fastercited by 0
  10. 2023Interactively Teaching an Inverse Reinforcement Learner with Limited Feedbackcited by 0
  11. 2023Optimally Teaching a Linear Behavior Cloning Agentcited by 0
  12. 2026Interactive Inverse Reinforcement Learning of Interaction Scenarios via Bi-level Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Inverse Reinforcement Learning without Reinforcement Learning
  2. 2024Hybrid Inverse Reinforcement Learning
  3. 2023Data-Driven Inverse Reinforcement Learning for Expert-Learner Zero-Sum Games
  4. 2020Inverse Reinforcement Learning from a Gradient-based Learner
  5. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
  6. 2024Inverse Reinforcement Learning by Estimating Expertise of Demonstrators
  7. 2021Inverse Reinforcement Learning in a Continuous State Space with Formal Guarantees
  8. 2024The Virtues of Pessimism in Inverse Reinforcement Learning