mlsanityData to 14 Oct 2026

Proximal Curriculum for Reinforcement Learning Agents

arXiv 2304.12877 ↗First posted 25 Apr 2023cs.LGcited by 20

Not on the map. Why?

Abstract (arXiv)

We consider the problem of curriculum design for reinforcement learning (RL) agents in contextual multi-task settings. Existing techniques on automatic curriculum design typically require domain-specific hyperparameter tuning or have limited theoretical underpinnings. To tackle these limitations, we design our curriculum strategy, ProCuRL, inspired by the pedagogical concept of Zone of Proximal Development (ZPD). ProCuRL captures the intuition that learning progress is maximized when picking tasks that are neither too hard nor too easy for the learner. We mathematically derive ProCuRL by analyzing two simple learning settings. We also present a practical variant of ProCuRL that can be directly integrated with deep RL frameworks with minimal hyperparameter tuning. Experimental results on a variety of domains demonstrate the effectiveness of our curriculum strategy over state-of-the-art baselines in accelerating the training process of deep RL agents.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 42 of its 70 references that are in the map. First posted on arXiv 25 Apr 2023.

Builds on

42 of 70 references.

References · 42

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2017Hindsight Experience Replaycited by 1,136
  4. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  5. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  6. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  7. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  8. 2017Automated Curriculum Learning for Neural Networkscited by 257
  9. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  10. 2017Teacher-Student Curriculum Learningcited by 210
  11. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 198
  12. 2020Prioritized Level Replaycited by 198
30 more
  1. 2022Evolving Curricula with Regret-Based Environment Designcited by 173
  2. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 167
  3. 2019Goal-conditioned Imitation Learningcited by 162
  4. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 150
  5. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 141
  6. 2018Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networkscited by 114
  7. 2021Replay-Guided Adversarial Environment Designcited by 111
  8. 2014Learning to Executecited by 106
  9. 2019Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Datacited by 104
  10. 2020Automatic Curriculum Learning through Value Disagreementcited by 87
  11. 2018Leveraging Grammar and Reinforcement Learning for Neural Program Synthesiscited by 84
  12. 2018Learning Montezuma's Revenge from a Single Demonstrationcited by 80
  13. 2020Safe Reinforcement Learning via Curriculum Inductioncited by 74
  14. 2018An Overview of Machine Teachingcited by 63
  15. 2013Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robotscited by 58
  16. 2020Goal-GAN: Multimodal Trajectory Prediction Based on Goal Position Estimationcited by 57
  17. 2020Self-Paced Deep Reinforcement Learningcited by 50
  18. 2017Iterative Machine Teachingcited by 41
  19. 2018Learning Curriculum Policies for Reinforcement Learningcited by 35
  20. 2018Theory of Curriculum Learning, with Convex Loss Functionscited by 33
  21. 2022Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptationcited by 28
  22. 2019Automated curricula through setter-solver interactionscited by 26
  23. 2019Self-Paced Contextual Reinforcement Learningcited by 26
  24. 2021Self-Paced Context Evaluation for Contextual Reinforcement Learningcited by 25
  25. 2019Interactive Teaching Algorithms for Inverse Reinforcement Learningcited by 24
  26. 2021A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learningcited by 22
  27. 2018Backplay: "Man muss immer umkehren"cited by 21
  28. 2019Reinforcement Learning without Ground-Truth Statecited by 15
  29. 2021Curriculum Design for Teaching via Demonstrations: Theory and Applicationscited by 9
  30. 2017Optimal Cooperative Inferencecited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 8; 2026: 7.

20 papers.

Sort:

By area:

  1. 2025Self-Evolving Curriculum for LLM Reasoningcited by 112
  2. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 45
  3. 2024No Regrets: Investigating and Improving Regret Approximations for Curriculum Discoverycited by 31
  4. 2025Learning to Reason at the Frontier of Learnabilitycited by 23
  5. 2025An Optimisation Framework for Unsupervised Environment Designcited by 10
  6. 2026Actor-Curator: Co-adaptive Curriculum Learning via Policy-Improvement Bandits for RL Post-Trainingcited by 8
  7. 2024Proximal Curriculum with Task Correlations for Deep Reinforcement Learningcited by 7
  8. 2026Transferability for General Reasoning: An Automated Curriculum for Multi-Domain RLVRcited by 5
  9. 2025Unsupervised Partner Design Enables Robust Ad-hoc Teamworkcited by 3
  10. 2024Syllabus: Portable Curricula for Reinforcement Learning Agentscited by 2
  11. 2025Curriculum Design for Trajectory-Constrained Agent: Compressing Chain-of-Thought Tokens in LLMscited by 2
  12. 2026AdaGRPO: A Capability-Aware Adaptive Enhancement for Flow-based GRPOcited by 2
8 more
  1. 2025TRACED: Transition-aware Regret Approximation with Co-learnability for Environment Designcited by 1
  2. 2025Causal-Paced Deep Reinforcement Learningcited by 1
  3. 2026Improving Regret Approximation for Unsupervised Dynamic Environment Generationcited by 1
  4. 2026Scaling Curriculum Learning For Autonomous Drivingcited by 1
  5. 2023Learning Embeddings for Sequential Tasks Using Population of Agentscited by 0
  6. 2025Knowledge capture, adaptation and composition (KCAC): A framework for cross-task curriculum learning in robotic manipulationcited by 0
  7. 2026LLM-as-a-Tutor: Policy-Aware Prompt Adaptation for Non-Verifiable RLcited by 0
  8. 2026BoundaryEvo: Data-Model Co-Evolution at the Capability Boundary for Long-Chain Reasoning Distillationcited by 0