mlsanityData to 14 Oct 2026

Teacher-student curriculum learning for reinforcement learning

arXiv 2210.17368 ↗First posted 31 Oct 2022cs.LG cs.AIcited by 2

On the map: Knowledge distillation → On-policy and self-distillation for LLMs

Abstract (arXiv)

Reinforcement learning (rl) is a popular paradigm for sequential decision making problems. The past decade's advances in rl have led to breakthroughs in many challenging domains such as video games, board games, robotics, and chip design. The sample inefficiency of deep reinforcement learning methods is a significant obstacle when applying rl to real-world problems. Transfer learning has been applied to reinforcement learning such that the knowledge gained in one task can be applied when training in a new task. Curriculum learning is concerned with sequencing tasks or data samples such that knowledge can be transferred between those tasks to learn a target task that would otherwise be too difficult to solve. Designing a curriculum that improves sample efficiency is a complex problem. In this thesis, we propose a teacher-student curriculum learning setting where we simultaneously train a teacher that selects tasks for the student while the student learns how to solve the selected task. Our method is independent of human domain knowledge and manual curriculum design. We evaluated our methods on two reinforcement learning benchmarks: grid world and the challenging Google Football environment. With our method, we can improve the sample efficiency and generality of the student compared to tabula-rasa reinforcement learning.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 24 of its 63 references that are in the map. First posted on arXiv 31 Oct 2022.

Builds on

24 of 63 references.

References · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2017Hindsight Experience Replaycited by 1,136
  4. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  5. 2015Prioritized Experience Replaycited by 1,099
  6. 2019Solving Rubik's Cube with a Robot Handcited by 734
  7. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  8. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 303
  9. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 260
  10. 2017Teacher-Student Curriculum Learningcited by 210
  11. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 198
  12. 2020Prioritized Level Replaycited by 198
12 more
  1. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 167
  2. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 119
  3. 2020Revisiting Design Choices in Proximal Policy Optimizationcited by 38
  4. 2018Learning Curriculum Policies for Reinforcement Learningcited by 35
  5. 2021Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraftcited by 35
  6. 2016Strategic Attentive Writer for Learning Macro-Actionscited by 30
  7. 2018StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer Learningcited by 25
  8. 2020Sample Efficient Reinforcement Learning through Learning from Demonstrations in Minecraftcited by 15
  9. 2019An Optimization Framework for Task Sequencing in Curriculum Learningcited by 8
  10. 2019Curriculum Learning for Cumulative Return Maximizationcited by 4
  11. 2018ScreenerNet: Learning Self-Paced Curriculum for Deep Neural Networkscited by 2
  12. 2020Learning Value Functions in Deep Policy Gradients using Residual Variancecited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1.

2 papers.

  1. 2023Continual Driving Policy Optimization with Closed-Loop Individualized Curriculacited by 4
  2. 2024Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Reviewcited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2021DCUR: Data Curriculum for Teaching via Samples with Reinforcement Learning
  2. 2025Probabilistic Curriculum Learning for Goal-Based Reinforcement Learning
  3. 2023TGRL: An Algorithm for Teacher Guided Reinforcement Learning
  4. 2024Curriculum Reinforcement Learning for Complex Reward Functions
  5. 2024Syllabus: Portable Curricula for Reinforcement Learning Agents
  6. 2025Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning
  7. 2024Grounded Curriculum Learning
  8. 2022Task Phasing: Automated Curriculum Learning from Demonstrations