mlsanityData to 14 Oct 2026

TEACH: Temporal Variance-Driven Curriculum for Reinforcement Learning

arXiv 2512.22824 ↗First posted 28 Dec 2025cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Reinforcement Learning (RL) has achieved significant success in solving single-goal tasks. However, uniform goal selection often results in sample inefficiency in multi-goal settings where agents must learn a universal goal-conditioned policy. Inspired by the adaptive and structured learning processes observed in biological systems, we propose a novel Student-Teacher learning paradigm with a Temporal Variance-Driven Curriculum to accelerate Goal-Conditioned RL. In this framework, the teacher module dynamically prioritizes goals with the highest temporal variance in the policy's confidence score, parameterized by the state-action value (Q) function. The teacher provides an adaptive and focused learning signal by targeting these high-uncertainty goals, fostering continual and efficient progress. We establish a theoretical connection between the temporal variance of Q-values and the evolution of the policy, providing insights into the method's underlying principles. Our approach is algorithm-agnostic and integrates seamlessly with existing RL frameworks. We demonstrate this through evaluation across 11 diverse robotic manipulation and maze navigation tasks. The results show consistent and notable improvements over state-of-the-art curriculum learning and goal-selection methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 33 of its 55 references that are in the map. First posted on arXiv 28 Dec 2025.

Builds on

33 of 55 references.

References · 33

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2017Hindsight Experience Replaycited by 1,136
  3. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  4. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  5. 2017Robust Adversarial Reinforcement Learningcited by 451
  6. 2017Deep Reinforcement Learning: An Overviewcited by 333
  7. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  8. 2018Distributed Prioritized Experience Replaycited by 269
  9. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 260
  10. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  11. 2017Automated Curriculum Learning for Neural Networkscited by 257
  12. 2017Teacher-Student Curriculum Learningcited by 210
21 more
  1. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 198
  2. 2020Prioritized Level Replaycited by 198
  3. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 182
  4. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 174
  5. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 150
  6. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 141
  7. 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 138
  8. 2021Discovering and Achieving Goals via World Modelscited by 131
  9. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 119
  10. 2018Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networkscited by 114
  11. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 102
  12. 2020Automatic Curriculum Learning through Value Disagreementcited by 87
  13. 2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 82
  14. 2018Deep Drone Racing: Learning Agile Flight in Dynamic Environmentscited by 34
  15. 2019Automated curricula through setter-solver interactionscited by 26
  16. 2021Self-Paced Context Evaluation for Contextual Reinforcement Learningcited by 25
  17. 2021A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learningcited by 22
  18. 2020Guided Exploration with Proximal Policy Optimization using a Single Demonstrationcited by 19
  19. 2022Understanding the Complexity Gains of Single-Task RL with a Curriculumcited by 17
  20. 2021Curriculum Design for Teaching via Demonstrations: Theory and Applicationscited by 9
  21. 2018Deep RTS: A Game Environment for Deep Reinforcement Learning in Real-Time Strategy Gamescited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Probabilistic Curriculum Learning for Goal-Based Reinforcement Learning
  2. 2023CQM: Curriculum Reinforcement Learning with a Quantized World Model
  3. 2025Trajectory First: A Curriculum for Discovering Diverse Policies
  4. 2025Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning
  5. 2026ACDC: Adaptive Curriculum Planning with Dynamic Contrastive Control for Goal-Conditioned Reinforcement Learning in Robotic Manipulation
  6. 2025Causal-Paced Deep Reinforcement Learning
  7. 2026RSIPO: Recursive Self-Improving Policy Optimization for Agentic Reinforcement Learning via Skill-Policy Co-Evolution
  8. 2022Teacher-student curriculum learning for reinforcement learning