mlsanityData to 14 Oct 2026

Causally Aligned Curriculum Learning

arXiv 2503.16799 ↗First posted 21 Mar 2025cs.LG cs.AIcited by 6

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

A pervasive challenge in Reinforcement Learning (RL) is the "curse of dimensionality" which is the exponential growth in the state-action space when optimizing a high-dimensional target task. The framework of curriculum learning trains the agent in a curriculum composed of a sequence of related and more manageable source tasks. The expectation is that when some optimal decision rules are shared across source tasks and the target task, the agent could more quickly pick up the necessary skills to behave optimally in the environment, thus accelerating the learning process. However, this critical assumption of invariant optimal decision rules does not necessarily hold in many practical applications, specifically when the underlying environment contains unobserved confounders. This paper studies the problem of curriculum RL through causal lenses. We derive a sufficient graphical condition characterizing causally aligned source tasks, i.e., the invariance of optimal decision rules holds. We further develop an efficient algorithm to generate a causally aligned curriculum, provided with qualitative causal knowledge of the target task. Finally, we validate our proposed methodology through experiments in discrete and continuous confounded tasks with pixel observations.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 30 of its 65 references that are in the map. First posted on arXiv 21 Mar 2025.

Builds on

30 of 65 references.

References · 30

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Hindsight Experience Replaycited by 1,136
  3. 2018Exploration by Random Network Distillationcited by 993
  4. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  5. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  6. 2019Solving Rubik's Cube with a Robot Handcited by 734
  7. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  8. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  9. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  10. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  11. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 212
  12. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 203
18 more
  1. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 198
  2. 2020Prioritized Level Replaycited by 198
  3. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 192
  4. 2022Evolving Curricula with Regret-Based Environment Designcited by 173
  5. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 150
  6. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 141
  7. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 119
  8. 2020Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutionscited by 106
  9. 2018Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generationcited by 81
  10. 2018Learning Montezuma's Revenge from a Single Demonstrationcited by 80
  11. 2021On the Expressivity of Markov Rewardcited by 71
  12. 2019Increasing Generality in Machine Learning through Procedural Content Generationcited by 59
  13. 2013Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robotscited by 58
  14. 2022Causal Discovery in Heterogeneous Environments Under the Sparse Mechanism Shift Hypothesiscited by 55
  15. 2022Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptationcited by 28
  16. 2019Automated curricula through setter-solver interactionscited by 26
  17. 2022Causality-driven Hierarchical Structure Discovery for Reinforcement Learningcited by 24
  18. 2023Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generationcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2025AURA: Autonomous Upskilling with Retrieval-Augmented Agentscited by 2
  2. 2026Causal Flow Q-Learning for Robust Offline Reinforcement Learningcited by 2
  3. 2025Causal-Paced Deep Reinforcement Learningcited by 1
  4. 2026Active Curriculum Refinement for Reinforcement Learningcited by 1
  5. 2025Hierarchical Reinforcement Learning with Targeted Causal Interventionscited by 0
  6. 2026Level Up: Defining and Exploiting Transitional Problems for Curriculum Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Proximal Curriculum with Task Correlations for Deep Reinforcement Learning
  2. 2023CQM: Curriculum Reinforcement Learning with a Quantized World Model
  3. 2021Curriculum Design for Teaching via Demonstrations: Theory and Applications
  4. 2026An Introduction to Causal Reinforcement Learning
  5. 2025Curriculum-RLAIF: Curriculum Alignment with Reinforcement Learning from AI Feedback
  6. 2025Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning
  7. 2025Unifying Causal Reinforcement Learning: Survey, Taxonomy, Algorithms and Applications
  8. 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoning