mlsanityData to 14 Oct 2026

Syllabus: Portable Curricula for Reinforcement Learning Agents

arXiv 2411.11318 ↗First posted 18 Nov 2024cs.AIcited by 2

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Curriculum learning has been a quiet, yet crucial component of many high-profile successes of reinforcement learning. Despite this, it is still a niche topic that is not directly supported by any of the major reinforcement learning libraries. These methods can improve the capabilities and generalization of RL agents, but often require complex changes to training code. We introduce Syllabus, a portable curriculum learning library, as a solution to this problem. Syllabus provides a universal API for curriculum learning, modular implementations of popular automatic curriculum learning methods, and infrastructure that allows them to be easily integrated with asynchronous training code in nearly any RL library. Syllabus provides a minimal API for core curriculum learning components, making it easier to design new algorithms and adapt existing ones to new environments. We demonstrate this by evaluating the algorithms in Syllabus on several new environments, each using agents written in a different RL library. We present the first examples of automatic curriculum learning in NetHack and Neural MMO, two of the most challenging RL benchmarks, and find evidence that existing methods do not directly transfer to complex new environments. Syllabus can be found at https://github.com/RyanNavillus/Syllabus.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 71 of its 99 references that are in the map. First posted on arXiv 18 Nov 2024.

Builds on

71 of 99 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024No Regrets: Investigating and Improving Regret Approximations for Curriculum Discovery27 Aug
  2. 2024RLeXplore: Accelerating Research in Intrinsically-Motivated Reinforcement Learning29 May
  3. 2024Refining Minimax Regret for Unsupervised Environment Design19 Feb
  4. 2024Open RL Benchmark: Comprehensive Tracked Experiments for Reinforcement Learning5 Feb

Other references · 67

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  5. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  6. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  7. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,070
  8. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  9. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  10. 2019Solving Rubik's Cube with a Robot Handcited by 734
  11. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  12. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 527
38 more
  1. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  2. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  3. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  4. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  5. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 268
  6. 2020Transfer Learning in Deep Reinforcement Learning: A Surveycited by 265
  7. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  8. 2017Count-Based Exploration with Neural Density Modelscited by 257
  9. 2017Automated Curriculum Learning for Neural Networkscited by 257
  10. 2020PettingZoo: Gym for Multi-Agent Reinforcement Learningcited by 220
  11. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 215
  12. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 200
  13. 2020The NetHack Learning Environmentcited by 198
  14. 2020Prioritized Level Replaycited by 198
  15. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 192
  16. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 187
  17. 2022Evolving Curricula with Regret-Based Environment Designcited by 173
  18. 2019Active Domain Randomizationcited by 170
  19. 2016Deep Reinforcement Learning from Self-Play in Imperfect-Information Gamescited by 158
  20. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 150
  21. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 141
  22. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 138
  23. 2020Phasic Policy Gradientcited by 123
  24. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 119
  25. 2021Replay-Guided Adversarial Environment Designcited by 111
  26. 2020Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutionscited by 106
  27. 2020Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learningcited by 93
  28. 2018Learning Montezuma's Revenge from a Single Demonstrationcited by 80
  29. 2024OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Codecited by 74
  30. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 72
  31. 2023OMNI: Open-endedness via Models of human Notions of Interestingnesscited by 70
  32. 2019Autocurricula and the Emergence of Innovation from Social Interaction: A Manifesto for Multi-Agent Intelligence Researchcited by 61
  33. 2023XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAXcited by 55
  34. 2022Exploration via Elliptical Episodic Bonusescited by 51
  35. 2019Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agentscited by 44
  36. 2019TorchBeast: A PyTorch Platform for Distributed RLcited by 41
  37. 2021On Bonus-Based Exploration Methods in the Arcade Learning Environmentcited by 37
  38. 2021Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraftcited by 35

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Scaling Curriculum Learning For Autonomous Drivingcited by 1
  2. 2026Curriculum Generation under Structured Parametric Environments for Robust Navigation Policiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Probabilistic Curriculum Learning for Goal-Based Reinforcement Learning
  2. 2025Self-Evolving Curriculum for LLM Reasoning
  3. 2026RLLBC-Lib: An Educational Code Library for Reinforcement Learning and Learning-Based Control
  4. 2022Teacher-student curriculum learning for reinforcement learning
  5. 2024TeaMs-RL: Teaching LLMs to Generate Better Instruction Datasets via Reinforcement Learning
  6. 2025DUMP: Automated Distribution-Level Curriculum Learning for RL-based LLM Post-training
  7. 2025Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning
  8. 2024GLIDE-RL: Grounded Language Instruction through DEmonstration in RL