mlsanityData to 14 Oct 2026

On the Benefit of Optimal Transport for Curriculum Reinforcement Learning

arXiv 2309.14091 ↗First posted 25 Sep 2023cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Curriculum reinforcement learning (CRL) allows solving complex tasks by generating a tailored sequence of learning tasks, starting from easy ones and subsequently increasing their difficulty. Although the potential of curricula in RL has been clearly shown in various works, it is less clear how to generate them for a given learning environment, resulting in various methods aiming to automate this task. In this work, we focus on framing curricula as interpolations between task distributions, which has previously been shown to be a viable approach to CRL. Identifying key issues of existing methods, we frame the generation of a curriculum as a constrained optimal transport problem between task distributions. Benchmarks show that this way of curriculum generation can improve upon existing CRL methods, yielding high performance in various tasks with different characteristics.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 41 of its 87 references that are in the map. First posted on arXiv 25 Sep 2023.

Builds on

41 of 87 references.

References · 41

  1. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  2. 2017Hindsight Experience Replaycited by 1,136
  3. 2019Solving Rubik's Cube with a Robot Handcited by 734
  4. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 641
  5. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  6. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  7. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  8. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  9. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  10. 2017Automated Curriculum Learning for Neural Networkscited by 257
  11. 2015Contextual Markov Decision Processescited by 220
  12. 2019Generalized Sliced Wasserstein Distancescited by 208
29 more
  1. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 198
  2. 2020Prioritized Level Replaycited by 198
  3. 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 195
  4. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 141
  5. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 119
  6. 2018Count-Based Exploration with the Successor Representationcited by 112
  7. 2021Replay-Guided Adversarial Environment Designcited by 111
  8. 2020When Do Curricula Work?cited by 106
  9. 2019Wasserstein Weisfeiler-Lehman Graph Kernelscited by 104
  10. 2020Automatic Curriculum Learning through Value Disagreementcited by 87
  11. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  12. 2019Exploration via Hindsight Goal Generationcited by 54
  13. 2020Self-Paced Deep Reinforcement Learningcited by 50
  14. 2021Cross-Domain Imitation Learning via Optimal Transportcited by 49
  15. 2021Adversarial Intrinsic Motivation for Reinforcement Learningcited by 40
  16. 2018Policy Optimization as Wasserstein Gradient Flowscited by 34
  17. 2018Theory of Curriculum Learning, with Convex Loss Functionscited by 33
  18. 2020Continuous Regularized Wasserstein Barycenterscited by 32
  19. 2022Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptationcited by 28
  20. 2017Learning Wasserstein Embeddingscited by 26
  21. 2019Automated curricula through setter-solver interactionscited by 26
  22. 2019Self-Paced Contextual Reinforcement Learningcited by 26
  23. 2021Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problemscited by 26
  24. 2021Self-Paced Context Evaluation for Contextual Reinforcement Learningcited by 25
  25. 2021A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learningcited by 22
  26. 2021TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RLcited by 18
  27. 2022Understanding the Complexity Gains of Single-Task RL with a Curriculumcited by 17
  28. 2023Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generationcited by 15
  29. 2022Combining Reinforcement Learning and Optimal Transport for the Traveling Salesman Problemcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 2.

6 papers.

  1. 2024Acquiring Diverse Skills using Curriculum Reinforcement Learning with Mixture of Expertscited by 14
  2. 2025Optimizing Distributional Geometry Alignment with Optimal Transport for Generative Dataset Distillationcited by 5
  3. 2025Curriculum-Adapted Robust Reinforcement Learning for UAV Deconfliction in Adversarial Environmentscited by 1
  4. 2025Causal-Paced Deep Reinforcement Learningcited by 1
  5. 2026Geometry-Aware Dataset Condensation for Diffusion Model Trainingcited by 1
  6. 2026Revisiting On-policy Adversarial Black-Box Distillation: Calibrating Groupwise Reward Geometry for Effective Advantage Constructioncited by 0