mlsanityData to 7 Oct 2026

Self-Paced Context Evaluation for Contextual Reinforcement Learning

arXiv 2106.05110 ↗First posted 9 Jun 2021cs.LGcited by 25

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning (RL) has made a lot of advances for solving a single problem in a given environment; but learning policies that generalize to unseen variations of a problem remains challenging. To improve sample efficiency for learning on such instances of a problem domain, we present Self-Paced Context Evaluation (SPaCE). Based on self-paced learning, spc automatically generates task curricula online with little computational overhead. To this end, SPaCE leverages information contained in state values during training to accelerate and improve training performance as well as generalization capabilities to new instances from the same problem domain. Nevertheless, SPaCE is independent of the problem domain at hand and can be applied on top of any RL agent with state-value function approximation. We demonstrate SPaCE's ability to speed up learning of different value-based RL agents on two environments, showing better generalization capabilities and up to 10x faster learning compared to naive approaches such as round robin or SPDRL, as the closest state-of-the-art approach.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 20 of its 37 references that are in the map. First posted on arXiv 9 Jun 2021.

Builds on

20 of 37 references.

References · 20

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  6. 2017Hindsight Experience Replaycited by 1,096
  7. 2019Solving Rubik's Cube with a Robot Handcited by 699
  8. 2015Contextual Markov Decision Processescited by 213
  9. 2017Teacher-Student Curriculum Learningcited by 197
  10. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 185
  11. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 139
  12. 2019Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Datacited by 103
8 more
  1. 2020Automatic Curriculum Learning through Value Disagreementcited by 86
  2. 2020Learning to Continually Learncited by 82
  3. 2020Safe Reinforcement Learning via Curriculum Inductioncited by 73
  4. 2020Goal-GAN: Multimodal Trajectory Prediction Based on Goal Position Estimationcited by 57
  5. 2017Markov Decision Processes with Continuous Side Informationcited by 54
  6. 2020Self-Paced Deep Reinforcement Learningcited by 49
  7. 2020Trajectory-wise Multiple Choice Learning for Dynamics Generalization in Reinforcement Learningcited by 34
  8. 2020Learning Heuristic Selection with Dynamic Algorithm Configurationcited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 4; 2023: 8; 2024: 2; 2025: 6; 2026: 4.

25 papers.

Sort:

By area:

  1. 2022Evolving Curricula with Regret-Based Environment Designcited by 166
  2. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  3. 2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 41
  4. 2022Automated Dynamic Algorithm Configurationcited by 37
  5. 2023MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learningcited by 30
  6. 2021CARL: A Benchmark for Contextual and Adaptive Reinforcement Learningcited by 21
  7. 2023Proximal Curriculum for Reinforcement Learning Agentscited by 19
  8. 2023Reward-Free Curricula for Training Robust World Modelscited by 13
  9. 2023Train Hard, Fight Easy: Robust Meta Reinforcement Learningcited by 11
  10. 2023On the Benefit of Optimal Transport for Curriculum Reinforcement Learningcited by 6
  11. 2024Proximal Curriculum with Task Correlations for Deep Reinforcement Learningcited by 6
  12. 2023Experimentation Platforms Meet Reinforcement Learning: Bayesian Sequential Decision-Making for Continuous Monitoringcited by 3
13 more
  1. 2025cMALC-D: Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blendingcited by 2
  2. 2025Curriculum Design for Trajectory-Constrained Agent: Compressing Chain-of-Thought Tokens in LLMscited by 2
  3. 2026Structure Detection for Contextual Reinforcement Learningcited by 2
  4. 2023Reward-Machine-Guided, Self-Paced Reinforcement Learningcited by 1
  5. 2025TRACED: Transition-aware Regret Approximation with Co-learnability for Environment Designcited by 1
  6. 2025Distributionally Robust Self Paced Curriculum Reinforcement Learningcited by 1
  7. 2026Contextual Intelligence The Next Leap for Reinforcement Learningcited by 1
  8. 2023Learning Embeddings for Sequential Tasks Using Population of Agentscited by 0
  9. 2024Online Optimization of Curriculum Learning Schedules using Evolutionary Optimizationcited by 0
  10. 2025Observations Meet Actions: Learning Control-Sufficient Representations for Robust Policy Generalizationcited by 0
  11. 2025TEACH: Temporal Variance-Driven Curriculum for Reinforcement Learningcited by 0
  12. 2026Self Paced Gaussian Contextual Reinforcement Learningcited by 0
  13. 2026Safe and Robust Neural Policy Learning with Statistical Verification for Sim-to-Real Deployment in Roboticscited by 0