mlsanityData to 7 Oct 2026

Curriculum Learning as Transport: Understanding Curricula with Wasserstein Geodesics

arXiv 2609.09099 ↗First posted 8 Sep 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Curriculum learning is governed by several coupled design choices---how difficulty is defined, how examples are ordered, how much exposure each level receives, and how quickly training moves across levels---making it hard to isolate what actually helps. We present Wasserstein curriculum paths, a simple transport-based framework that decouples these factors by representing curricula as trajectories of training distributions over discrete difficulty levels. Across a calibrated synthetic suite with 12 tasks and 33 difficulty axes, we use this framework to isolate the effects of ordering, matched exposure, endpoint smoothness, and pacing under fixed training budgets. We find that curriculum effects are strongly context-dependent: no single strategy dominates across tasks, difficulty axes, and budgets, and curricula mainly change where a fixed budget is spent most effectively. Within this framework, easy-to-hard ordering improves hard-level performance relative to exposure-matched static sampling, showing that the benefit is not explained by cumulative exposure alone. We further show that endpoint smoothness and pacing substantially affect where along the difficulty spectrum a curriculum is effective. Finally, we show that the same transport view naturally supports extensions to learned pacing through geometry and to structured difficulty spaces beyond one-dimensional orderings.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 36 references that are in the map. First posted on arXiv 8 Sep 2026.

Builds on

25 of 36 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  5. 2018Computational Optimal Transportcited by 765
  6. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  7. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  8. 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
  9. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 423
  10. 2023DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretrainingcited by 385
  11. 2019On The Power of Curriculum Learning in Training Deep Networkscited by 285
  12. 2017Automated Curriculum Learning for Neural Networkscited by 251
13 more
  1. 2019Competence-based Curriculum Learning for Neural Machine Translationcited by 201
  2. 2017Teacher-Student Curriculum Learningcited by 197
  3. 2024RegMix: Data Mixture as Regression for Language Model Pre-trainingcited by 172
  4. 2020When Do Curricula Work?cited by 105
  5. 2023DoGE: Domain Reweighting with Generalization Estimationcited by 87
  6. 2025SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyondcited by 47
  7. 2024The Unreasonable Effectiveness of Easy Training Data for Hard Taskscited by 40
  8. 2022Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptationcited by 28
  9. 2025What Makes a Good Curriculum? Disentangling the Effects of Data Ordering on LLM Mathematical Reasoningcited by 10
  10. 2024A Psychology-based Unified Dynamic Framework for Curriculum Learningcited by 6
  11. 2025Optimizing Pre-Training Data Mixtures with Mixtures of Data Expert Modelscited by 5
  12. 2025Topic Over Source: The Key to Effective Data Mixing for Language Models Pre-trainingcited by 4
  13. 2025R&B: Domain Regrouping and Data Mixture Balancing for Efficient Foundation Model Trainingcited by 3

Built on by

Not cited yet.