mlsanityData to 7 Oct 2026

Curriculum Design for Teaching via Demonstrations: Theory and Applications

arXiv 2106.04696 ↗First posted 8 Jun 2021cs.LG cs.AIcited by 9

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

We consider the problem of teaching via demonstrations in sequential decision-making settings. In particular, we study how to design a personalized curriculum over demonstrations to speed up the learner's convergence. We provide a unified curriculum strategy for two popular learner models: Maximum Causal Entropy Inverse Reinforcement Learning (MaxEnt-IRL) and Cross-Entropy Behavioral Cloning (CrossEnt-BC). Our unified strategy induces a ranking over demonstrations based on a notion of difficulty scores computed w.r.t. the teacher's optimal policy and the learner's current policy. Compared to the state of the art, our strategy doesn't require access to the learner's internal dynamics and still enjoys similar convergence guarantees under mild technical conditions. Furthermore, we adapt our curriculum strategy to the setting where no teacher agent is present using task-specific difficulty scores. Experiments on a synthetic car driving environment and navigation-based environments demonstrate the effectiveness of our curriculum strategy.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 32 of its 70 references that are in the map. First posted on arXiv 8 Jun 2021.

Builds on

32 of 70 references.

Closest ancestors · 4

Same direction, newest first

  1. 2019Learner-aware Teaching: Inverse Reinforcement Learning with Preferences and Constraints2 Jun
  2. 2019Interactive Teaching Algorithms for Inverse Reinforcement Learning28 May
  3. 2018Teaching Inverse Reinforcement Learners via Features and Demonstrations21 Oct
  4. 2018Machine Teaching for Inverse Reinforcement Learning: Algorithms and Applications20 May

Other references · 28

  1. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
  2. 2018Attention, Learn to Solve Routing Problems!cited by 646
  3. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  4. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  5. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 350
  6. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  7. 2017Automated Curriculum Learning for Neural Networkscited by 251
  8. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 185
  9. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
  10. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 164
  11. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 139
  12. 2018Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networkscited by 113
16 more
  1. 2014Learning to Executecited by 105
  2. 2020When Do Curricula Work?cited by 105
  3. 2018An Overview of Machine Teachingcited by 63
  4. 2017Iterative Machine Teachingcited by 40
  5. 2018Learning Curriculum Policies for Reinforcement Learningcited by 34
  6. 2018Theory of Curriculum Learning, with Convex Loss Functionscited by 33
  7. 2017Repeated Inverse Reinforcement Learningcited by 29
  8. 2018Mix&Match - Agent Curricula for Reinforcement Learningcited by 26
  9. 2019On Learning Paradigms for the Travelling Salesman Problemcited by 24
  10. 2020Value Alignment Verificationcited by 20
  11. 2017Towards Black-box Iterative Machine Teachingcited by 19
  12. 2018Teaching Multiple Concepts to a Forgetful Learnercited by 17
  13. 2018Understanding the Role of Adaptivity in Machine Teaching: The Case of Version Space Learnerscited by 16
  14. 2019Preference-Based Batch and Sequential Teaching: Towards a Unified View of Modelscited by 12
  15. 2018Iterative Classroom Teachingcited by 8
  16. 2017Optimal Cooperative Inferencecited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 1; 2025: 1; 2026: 2.

9 papers.

  1. 2023Proximal Curriculum for Reinforcement Learning Agentscited by 19
  2. 2026Actor-Curator: Co-adaptive Curriculum Learning via Policy-Improvement Bandits for RL Post-Trainingcited by 7
  3. 2024Proximal Curriculum with Task Correlations for Deep Reinforcement Learningcited by 6
  4. 2022Task Phasing: Automated Curriculum Learning from Demonstrationscited by 2
  5. 2022Autonomous Assessment of Demonstration Sufficiency via Bayesian Inverse Reinforcement Learningcited by 1
  6. 2026Learning from Demonstrations via Capability-Aware Goal Samplingcited by 1
  7. 2023Learning Embeddings for Sequential Tasks Using Population of Agentscited by 0
  8. 2023Interactively Teaching an Inverse Reinforcement Learner with Limited Feedbackcited by 0
  9. 2025TEACH: Temporal Variance-Driven Curriculum for Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Automatic Curricula via Expert Demonstrations
  2. 2020Interaction-limited Inverse Reinforcement Learning
  3. 2024Adaptive Teaching in Heterogeneous Agents: Balancing Surprise in Sparse Reward Scenarios
  4. 2023Information Maximizing Curriculum: A Curriculum-Based Approach for Imitating Diverse Skills
  5. 2025Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning
  6. 2026Scaling Curriculum Learning For Autonomous Driving
  7. 2022Fast Lifelong Adaptive Inverse Reinforcement Learning from Demonstrations
  8. 2020Bridging the Imitation Gap by Adaptive Insubordination