mlsanityData to 7 Oct 2026

TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL

arXiv 2103.09815 ↗First posted 17 Mar 2021cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

Training autonomous agents able to generalize to multiple tasks is a key target of Deep Reinforcement Learning (DRL) research. In parallel to improving DRL algorithms themselves, Automatic Curriculum Learning (ACL) study how teacher algorithms can train DRL agents more efficiently by adapting task selection to their evolving abilities. While multiple standard benchmarks exist to compare DRL agents, there is currently no such thing for ACL algorithms. Thus, comparing existing approaches is difficult, as too many experimental parameters differ from paper to paper. In this work, we identify several key challenges faced by ACL algorithms. Based on these, we present TeachMyAgent (TA), a benchmark of current ACL algorithms leveraging procedural task generation. It includes 1) challenge-specific unit-tests using variants of a procedural Box2D bipedal walker environment, and 2) a new procedural Parkour environment combining most ACL challenges, making it ideal for global performance assessment. We then use TeachMyAgent to conduct a comparative study of representative existing approaches, showcasing the competitiveness of some ACL algorithms that do not use expert knowledge. We also show that the Parkour environment remains an open problem. We open-source our environments, all studied ACL algorithms (collected from open-source code or re-implemented), and DRL students in a Python package available at https://github.com/flowersteam/TeachMyAgent.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 48 of its 80 references that are in the map. First posted on arXiv 17 Mar 2021.

Builds on

48 of 80 references.

References · 48

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2015Continuous control with deep reinforcement learningcited by 3,279
  5. 2016Density estimation using Real NVPcited by 2,029
  6. 2016OpenAI Gymcited by 1,684
  7. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  8. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  9. 2018Exploration by Random Network Distillationcited by 966
  10. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  11. 2019Solving Rubik's Cube with a Robot Handcited by 699
  12. 2017Wasserstein GANcited by 635
36 more
  1. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  2. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  3. 2018Quantifying Generalization in Reinforcement Learningcited by 345
  4. 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
  5. 2018Meta-Learning: A Surveycited by 254
  6. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  7. 2017Automated Curriculum Learning for Neural Networkscited by 251
  8. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 238
  9. 2017Teacher-Student Curriculum Learningcited by 197
  10. 2020Prioritized Level Replaycited by 190
  11. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 182
  12. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 171
  13. 2018A Study on Overfitting in Deep Reinforcement Learningcited by 170
  14. 2019Active Domain Randomizationcited by 161
  15. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 141
  16. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 138
  17. 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 136
  18. 2019Behaviour Suite for Reinforcement Learningcited by 129
  19. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 113
  20. 2020Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutionscited by 103
  21. 2020Automatic Curriculum Learning through Value Disagreementcited by 86
  22. 2020Learning with AMIGo: Adversarially Motivated Intrinsic Goalscited by 86
  23. 2018Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generationcited by 80
  24. 2017Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learningcited by 79
  25. 2018CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learningcited by 78
  26. 2018Learning Montezuma's Revenge from a Single Demonstrationcited by 76
  27. 2019Weight Agnostic Neural Networkscited by 71
  28. 2018Gotta Learn Fast: A New Benchmark for Generalization in RLcited by 59
  29. 2018How Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experimentscited by 56
  30. 2018Reinforcement Learning for Improving Agent Designcited by 55
  31. 2020Self-Paced Deep Reinforcement Learningcited by 49
  32. 2019Unsupervised Curricula for Visual Meta-Reinforcement Learningcited by 33
  33. 2019Automated curricula through setter-solver interactionscited by 26
  34. 2020GRIMGEP: Learning Progress for Robust Goal Sampling in Visual Deep Reinforcement Learningcited by 9
  35. 2017Recurrent Deterministic Policy Gradient Method for Bipedal Locomotion on Rough Terrain Challengecited by 7
  36. 2020Meta Automatic Curriculum Learningcited by 6

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 3; 2023: 5; 2024: 4; 2025: 4.

18 papers.

Sort:

By area:

  1. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  2. 2022Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptationcited by 28
  3. 2021CARL: A Benchmark for Contextual and Adaptive Reinforcement Learningcited by 21
  4. 2021Tuning Mixed Input Hyperparameters on the Fly for Efficient Population Based AutoRLcited by 8
  5. 2025MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spacescited by 7
  6. 2023On the Benefit of Optimal Transport for Curriculum Reinforcement Learningcited by 6
  7. 2024Proximal Curriculum with Task Correlations for Deep Reinforcement Learningcited by 6
  8. 2022GriddlyJS: A Web IDE for Reinforcement Learningcited by 4
  9. 2023Improving Generalization in Game Agents with Data Augmentation in Imitation Learningcited by 4
  10. 2023Continual Driving Policy Optimization with Closed-Loop Individualized Curriculacited by 4
  11. 2023Automatic Curriculum Learning with Gradient Reward Signalscited by 3
  12. 2025Heterogeneous Adversarial Play in Interactive Environmentscited by 3
6 more
  1. 2024Syllabus: Portable Curricula for Reinforcement Learning Agentscited by 2
  2. 2023C-Procgen: Empowering Procgen with Controllable Contextscited by 1
  3. 2024Sample Efficient Myopic Exploration Through Multitask Reinforcement Learning with Diverse Taskscited by 1
  4. 2024Rethinking Teacher-Student Curriculum Learning through the Cooperative Mechanics of Experiencecited by 1
  5. 2025TRACED: Transition-aware Regret Approximation with Co-learnability for Environment Designcited by 1
  6. 2025Causal-Paced Deep Reinforcement Learningcited by 1