mlsanityData to 7 Oct 2026

Automatic Curriculum Learning For Deep RL: A Short Survey

arXiv 2003.04664 ↗First posted 10 Mar 2020cs.LG cs.AI stat.MLcited by 141

Not on the map. Why?

Abstract (arXiv)

Automatic Curriculum Learning (ACL) has become a cornerstone of recent successes in Deep Reinforcement Learning (DRL).These methods shape the learning trajectories of agents by challenging them with tasks adapted to their capacities. In recent years, they have been used to improve sample efficiency and asymptotic performance, to organize exploration, to encourage generalization or to solve sparse reward problems, among others. The ambition of this work is dual: 1) to present a compact and accessible introduction to the Automatic Curriculum Learning literature and 2) to draw a bigger picture of the current state of the art in ACL to encourage the cross-breeding of existing concepts and the emergence of new ideas.

Key facts. Cited by 141 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 26; 2023: 21; 2024: 19; 2025: 30; 2026: 32.

141 papers.

Sort:

By area:

  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  2. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  3. 2022Evolving Curricula with Regret-Based Environment Designcited by 166
  4. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  5. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 111
  6. 2021Replay-Guided Adversarial Environment Designcited by 108
  7. 2025Self-Evolving Curriculum for LLM Reasoningcited by 107
  8. 2025STP: Self-play LLM Theorem Provers with Iterative Conjecturing and Provingcited by 77
  9. 2021Reinforcement learning for optimization of variational quantum circuit architecturescited by 76
  10. 2025Can Large Reasoning Models Self-Train?cited by 66
  11. 2025Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replaycited by 52
  12. 2022Improving Intrinsic Exploration with Language Abstractionscited by 49
38 more
  1. 2025RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environmentscited by 43
  2. 2021Visual Adversarial Imitation Learning using Variational Modelscited by 42
  3. 2022On the Robustness of Safe Reinforcement Learning under Observational Perturbationscited by 38
  4. 2023Augmenting Autotelic Agents with Large Language Modelscited by 29
  5. 2024No Regrets: Investigating and Improving Regret Approximations for Curriculum Discoverycited by 29
  6. 2025Training a Generally Curious Agentcited by 29
  7. 2022Learning to Solve Travelling Salesman Problem with Hardness-adaptive Curriculumcited by 27
  8. 2022Learning Dexterous Manipulation from Exemplar Object Trajectories and Pre-Graspscited by 27
  9. 2023ASP: Learn a Universal Neural Solver!cited by 27
  10. 2021Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problemscited by 25
  11. 2021Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systemscited by 24
  12. 2023The SocialAI School: Insights from Developmental Psychology Towards Artificial Socio-Cultural Agentscited by 24
  13. 2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 23
  14. 2023Latent Exploration for Reinforcement Learningcited by 21
  15. 2022General Intelligence Requires Rethinking Explorationcited by 20
  16. 2024A social path to human-like artificial intelligencecited by 20
  17. 2022CLUTR: Curriculum Learning via Unsupervised Task Representation Learningcited by 19
  18. 2023Proximal Curriculum for Reinforcement Learning Agentscited by 19
  19. 2024CurricuLLM: Automatic Task Curricula Design for Learning Complex Robot Skills using Large Language Modelscited by 19
  20. 2021TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RLcited by 18
  21. 2022Avalon: A Benchmark for RL Generalization Using Procedurally Generated Worldscited by 17
  22. 2022Embedding Synthetic Off-Policy Experience for Autonomous Driving via Zero-Shot Curriculacited by 17
  23. 2022Grounding Aleatoric Uncertainty for Unsupervised Environment Designcited by 16
  24. 2022It Takes Four to Tango: Multiagent Selfplay for Automatic Curriculum Generationcited by 14
  25. 2023Reward-Free Curricula for Training Robust World Modelscited by 13
  26. 2022Heterogeneous Multi-agent Zero-Shot Coordination by Coevolutioncited by 12
  27. 2022Automated Play-Testing Through RL Based Human-Like Play-Styles Generationcited by 11
  28. 2023minimax: Efficient Baselines for Autocurricula in JAXcited by 9
  29. 2024Discovering Sensorimotor Agency in Cellular Automata using Diversity Searchcited by 9
  30. 2024Jointly Training and Pruning CNNs via Learnable Agent Guidance and Alignmentcited by 9
  31. 2024Adversarial Environment Design via Regret-Guided Diffusion Modelscited by 9
  32. 2026HATS: Hardness-Aware Trajectory Synthesis for GUI Agentscited by 9
  33. 2026SEAL: Synergistic Co-Evolution of Agents and Learning Environmentscited by 9
  34. 2026Why AI systems don't learn and what to do about it: Lessons on autonomous learning from cognitive sciencecited by 8
  35. 2022Curriculum-based Asymmetric Multi-task Reinforcement Learningcited by 7
  36. 2024DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robotscited by 7
  37. 2025MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spacescited by 7
  38. 2026Digital Red Queen: Adversarial Program Evolution in Core War with LLMscited by 7

50 most cited of 141 All 141 (JSON)