mlsanityData to 7 Oct 2026

Proximal Curriculum for Reinforcement Learning Agents

arXiv 2304.12877 ↗First posted 25 Apr 2023cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

We consider the problem of curriculum design for reinforcement learning (RL) agents in contextual multi-task settings. Existing techniques on automatic curriculum design typically require domain-specific hyperparameter tuning or have limited theoretical underpinnings. To tackle these limitations, we design our curriculum strategy, ProCuRL, inspired by the pedagogical concept of Zone of Proximal Development (ZPD). ProCuRL captures the intuition that learning progress is maximized when picking tasks that are neither too hard nor too easy for the learner. We mathematically derive ProCuRL by analyzing two simple learning settings. We also present a practical variant of ProCuRL that can be directly integrated with deep RL frameworks with minimal hyperparameter tuning. Experimental results on a variety of domains demonstrate the effectiveness of our curriculum strategy over state-of-the-art baselines in accelerating the training process of deep RL agents.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 42 of its 70 references that are in the map. First posted on arXiv 25 Apr 2023.

Builds on

42 of 70 references.

References · 42

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2017Hindsight Experience Replaycited by 1,096
  4. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  5. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 350
  6. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 260
  7. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  8. 2017Automated Curriculum Learning for Neural Networkscited by 251
  9. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  10. 2017Teacher-Student Curriculum Learningcited by 197
  11. 2020Prioritized Level Replaycited by 190
  12. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 185
30 more
  1. 2022Evolving Curricula with Regret-Based Environment Designcited by 166
  2. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 164
  3. 2019Goal-conditioned Imitation Learningcited by 159
  4. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 141
  5. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 139
  6. 2018Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networkscited by 113
  7. 2021Replay-Guided Adversarial Environment Designcited by 108
  8. 2014Learning to Executecited by 105
  9. 2019Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Datacited by 103
  10. 2020Automatic Curriculum Learning through Value Disagreementcited by 86
  11. 2018Leveraging Grammar and Reinforcement Learning for Neural Program Synthesiscited by 83
  12. 2018Learning Montezuma's Revenge from a Single Demonstrationcited by 76
  13. 2020Safe Reinforcement Learning via Curriculum Inductioncited by 73
  14. 2018An Overview of Machine Teachingcited by 63
  15. 2020Goal-GAN: Multimodal Trajectory Prediction Based on Goal Position Estimationcited by 57
  16. 2013Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robotscited by 56
  17. 2020Self-Paced Deep Reinforcement Learningcited by 49
  18. 2017Iterative Machine Teachingcited by 40
  19. 2018Learning Curriculum Policies for Reinforcement Learningcited by 34
  20. 2018Theory of Curriculum Learning, with Convex Loss Functionscited by 33
  21. 2022Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptationcited by 28
  22. 2019Automated curricula through setter-solver interactionscited by 26
  23. 2019Self-Paced Contextual Reinforcement Learningcited by 26
  24. 2021Self-Paced Context Evaluation for Contextual Reinforcement Learningcited by 25
  25. 2019Interactive Teaching Algorithms for Inverse Reinforcement Learningcited by 24
  26. 2021A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learningcited by 21
  27. 2018Backplay: "Man muss immer umkehren"cited by 16
  28. 2019Reinforcement Learning without Ground-Truth Statecited by 15
  29. 2021Curriculum Design for Teaching via Demonstrations: Theory and Applicationscited by 9
  30. 2017Optimal Cooperative Inferencecited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 8; 2026: 6.

19 papers.

Sort:

By area:

  1. 2025Self-Evolving Curriculum for LLM Reasoningcited by 107
  2. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 40
  3. 2024No Regrets: Investigating and Improving Regret Approximations for Curriculum Discoverycited by 29
  4. 2025Learning to Reason at the Frontier of Learnabilitycited by 20
  5. 2025An Optimisation Framework for Unsupervised Environment Designcited by 9
  6. 2026Actor-Curator: Co-adaptive Curriculum Learning via Policy-Improvement Bandits for RL Post-Trainingcited by 7
  7. 2024Proximal Curriculum with Task Correlations for Deep Reinforcement Learningcited by 6
  8. 2026Transferability for General Reasoning: An Automated Curriculum for Multi-Domain RLVRcited by 4
  9. 2024Syllabus: Portable Curricula for Reinforcement Learning Agentscited by 2
  10. 2025Unsupervised Partner Design Enables Robust Ad-hoc Teamworkcited by 2
  11. 2025Curriculum Design for Trajectory-Constrained Agent: Compressing Chain-of-Thought Tokens in LLMscited by 2
  12. 2026AdaGRPO: A Capability-Aware Adaptive Enhancement for Flow-based GRPOcited by 2
7 more
  1. 2025TRACED: Transition-aware Regret Approximation with Co-learnability for Environment Designcited by 1
  2. 2025Causal-Paced Deep Reinforcement Learningcited by 1
  3. 2026Improving Regret Approximation for Unsupervised Dynamic Environment Generationcited by 1
  4. 2026Scaling Curriculum Learning For Autonomous Drivingcited by 1
  5. 2023Learning Embeddings for Sequential Tasks Using Population of Agentscited by 0
  6. 2025Knowledge capture, adaptation and composition (KCAC): A framework for cross-task curriculum learning in robotic manipulationcited by 0
  7. 2026LLM-as-a-Tutor: Policy-Aware Prompt Adaptation for Non-Verifiable RLcited by 0