# Learning Curriculum Policies for Reinforcement Learning

arXiv: https://arxiv.org/abs/1812.00285 · HTML: https://mlsanity.com/paper/1812.00285/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2018.

## Built on by (most cited first, 30 of 35)

- [Curriculum Learning: A Survey](https://mlsanity.com/paper/2101.10382/) (2021)
- [Autonomous Reinforcement Learning via Subgoal Curricula](https://mlsanity.com/paper/2107.12931/) (2021)
- [Environment Generation for Zero-Shot Compositional Reinforcement Learning](https://mlsanity.com/paper/2201.08896/) (2022)
- [A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learning](https://mlsanity.com/paper/2102.13176/) (2021)
- [Proximal Curriculum for Reinforcement Learning Agents](https://mlsanity.com/paper/2304.12877/) (2023)
- [RLOps: Development Life-cycle of Reinforcement Learning Aided Open RAN](https://mlsanity.com/paper/2111.06978/) (2021)
- [Robust Reinforcement Learning via Genetic Curriculum](https://mlsanity.com/paper/2202.08393/) (2022)
- [Cross-Episodic Curriculum for Transformer Agents](https://mlsanity.com/paper/2310.08549/) (2023)
- [Curriculum Design for Teaching via Demonstrations: Theory and Applications](https://mlsanity.com/paper/2106.04696/) (2021)
- [Variational Curriculum Reinforcement Learning for Unsupervised Discovery of Skills](https://mlsanity.com/paper/2310.19424/) (2023)
- [Relational Abstractions for Generalized Reinforcement Learning on Symbolic Problems](https://mlsanity.com/paper/2204.12665/) (2022)
- [Proximal Curriculum with Task Correlations for Deep Reinforcement Learning](https://mlsanity.com/paper/2405.02481/) (2024)
- [Learning to Transfer Role Assignment Across Team Sizes](https://mlsanity.com/paper/2204.12937/) (2022)
- [Towards Skilled Population Curriculum for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2302.03429/) (2023)
- [Investigating Value of Curriculum Reinforcement Learning in Autonomous Driving Under Diverse Road and Weather Conditions](https://mlsanity.com/paper/2103.07903/) (2021)
- [Learning Multi-Objective Curricula for Robotic Policy Learning](https://mlsanity.com/paper/2110.03032/) (2021)
- [Left Heavy Tails and the Effectiveness of the Policy and Value Networks in DNN-based best-first search for Sokoban Planning](https://mlsanity.com/paper/2206.14298/) (2022)
- [Heterogeneous Adversarial Play in Interactive Environments](https://mlsanity.com/paper/2510.18407/) (2025)
- [ReRec: Reasoning-Augmented LLM-based Recommendation Assistant via Reinforcement Fine-tuning](https://mlsanity.com/paper/2604.07851/) (2026)
- [Teacher-student curriculum learning for reinforcement learning](https://mlsanity.com/paper/2210.17368/) (2022)
- [A Comprehensive Study of Privacy Risks in Curriculum Learning](https://mlsanity.com/paper/2310.10124/) (2023)
- [Mitigating the Stability-Plasticity Dilemma in Adaptive Train Scheduling with Curriculum-Driven Continual DQN Expansion](https://mlsanity.com/paper/2408.09838/) (2024)
- [Probabilistic Curriculum Learning for Goal-Based Reinforcement Learning](https://mlsanity.com/paper/2504.01459/) (2025)
- [AURA: Autonomous Upskilling with Retrieval-Augmented Agents](https://mlsanity.com/paper/2506.02507/) (2025)
- [Reinforcement Teaching](https://mlsanity.com/paper/2204.11897/) (2022)
- [Multi-Phase Multi-Objective Dexterous Manipulation with Adaptive Hierarchical Curriculum](https://mlsanity.com/paper/2205.13441/) (2022)
- [Towards Transferring Tactile-based Continuous Force Control Policies from Simulation to Robot](https://mlsanity.com/paper/2311.07245/) (2023)
- [Direct Preference Optimization for Primitive-Enabled Hierarchical RL: A Bilevel Approach](https://mlsanity.com/paper/2411.00361/) (2024)
- [ForceGrip: Reference-Free Curriculum Learning for Realistic Grip Force Control in VR Hand Manipulation](https://mlsanity.com/paper/2503.08061/) (2025)
- [Interaction-limited Inverse Reinforcement Learning](https://mlsanity.com/paper/2007.00425/) (2020)

## Abstract (arXiv)

Curriculum learning in reinforcement learning is a training methodology that seeks to speed up learning of a difficult target task, by first training on a series of simpler tasks and transferring the knowledge acquired to the target task. Automatically choosing a sequence of such tasks (i.e. a curriculum) is an open problem that has been the subject of much recent work in this area. In this paper, we build upon a recent method for curriculum design, which formulates the curriculum sequencing problem as a Markov Decision Process. We extend this model to handle multiple transfer learning algorithms, and show for the first time that a curriculum policy over this MDP can be learned from experience. We explore various representations that make this possible, and evaluate our approach by learning curriculum policies for multiple agents in two different domains. The results show that our method produces curricula that can train agents to perform on a target task as fast or faster than existing methods.