mlsanityData to 7 Oct 2026

Learner-aware Teaching: Inverse Reinforcement Learning with Preferences and Constraints

arXiv 1906.00429 ↗First posted 2 Jun 2019cs.LG cs.AI stat.MLcited by 18

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Inverse reinforcement learning (IRL) enables an agent to learn complex behavior by observing demonstrations from a (near-)optimal policy. The typical assumption is that the learner's goal is to match the teacher's demonstrated behavior. In this paper, we consider the setting where the learner has its own preferences that it additionally takes into consideration. These preferences can for example capture behavioral biases, mismatched worldviews, or physical constraints. We study two teaching approaches: learner-agnostic teaching, where the teacher provides demonstrations from an optimal policy ignoring the learner's preferences, and learner-aware teaching, where the teacher accounts for the learner's preferences. We design learner-aware teaching algorithms and show that significant performance improvements can be achieved over learner-agnostic teaching.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 19 of its 52 references that are in the map. First posted on arXiv 2 Jun 2019.

Builds on

19 of 52 references.

Closest ancestors · 4

Same direction, newest first

  1. 2019Interactive Teaching Algorithms for Inverse Reinforcement Learning28 May
  2. 2019Risk-Aware Active Inverse Reinforcement Learning8 Jan
  3. 2018Teaching Inverse Reinforcement Learners via Features and Demonstrations21 Oct
  4. 2018Machine Teaching for Inverse Reinforcement Learning: Algorithms and Applications20 May

Other references · 15

  1. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  2. 2016Cooperative Inverse Reinforcement Learningcited by 298
  3. 2017Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 206
  4. 2017A Review on Bilevel Optimization: From Classical to Evolutionary Approaches and Applicationscited by 126
  5. 2018An Overview of Machine Teachingcited by 63
  6. 2014Near-Optimally Teaching the Crowd to Classifycited by 32
  7. 2018Teaching Categories to Human Learners with Visual Explanationscited by 30
  8. 2017Repeated Inverse Reinforcement Learningcited by 29
  9. 2015The Teaching Dimension of Linear Learnerscited by 19
  10. 2017Towards Black-box Iterative Machine Teachingcited by 19
  11. 2013Machine Teaching for Bayesian Learners in the Exponential Familycited by 18
  12. 2017Efficient Probabilistic Performance Bounds for Inverse Reinforcement Learningcited by 17
3 more
  1. 2018Teaching Multiple Concepts to a Forgetful Learnercited by 17
  2. 2018Understanding the Role of Adaptivity in Machine Teaching: The Case of Version Space Learnerscited by 16
  3. 2018Iterative Classroom Teachingcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 9; 2022: 3; 2023: 2; 2024: 0; 2025: 0; 2026: 2.

18 papers.

Sort:

By area:

  1. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
  2. 2021A Fully Single Loop Algorithm for Bilevel Optimization without Hessian Inversecited by 53
  3. 2020Robust Inverse Reinforcement Learning under Transition Dynamics Mismatchcited by 33
  4. 2019MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learningcited by 29
  5. 2021Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environmentscited by 26
  6. 2021Reinforcement Learning for Education: Opportunities and Challengescited by 25
  7. 2022Local Stochastic Bilevel Optimization with Momentum-Based Variance Reductioncited by 20
  8. 2023Identifiability and Generalizability in Constrained Inverse Reinforcement Learningcited by 16
  9. 2021Reinforcement Learning Under Algorithmic Triagecited by 15
  10. 2021Teaching an Active Learner with Contrastive Examplescited by 12
  11. 2021Fair Clustering Using Antidote Datacited by 10
  12. 2021Iterative Teaching by Label Synthesiscited by 10
6 more
  1. 2021Curriculum Design for Teaching via Demonstrations: Theory and Applicationscited by 9
  2. 2022Iterative Teaching by Data Hallucinationcited by 8
  3. 2026Fine-grained Analysis of Stability and Generalization for Stochastic Bilevel Optimizationcited by 3
  4. 2022On Stability and Generalization of Bilevel Optimization Problemcited by 1
  5. 2023Optimally Teaching a Linear Behavior Cloning Agentcited by 0
  6. 2026Multi-Modal, Multi-Environment Machine Teaching for Robust Reward Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning the Preferences of a Learning Agent
  2. 2020Inverse Reinforcement Learning from a Gradient-based Learner
  3. 2024Adaptive Teaching in Heterogeneous Agents: Balancing Surprise in Sparse Reward Scenarios
  4. 2023Interactively Teaching an Inverse Reinforcement Learner with Limited Feedback
  5. 2020Interaction-limited Inverse Reinforcement Learning
  6. 2025Generalizing Behavior via Inverse Reinforcement Learning with Closed-Form Reward Centroids
  7. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress
  8. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective