mlsanityData to 7 Oct 2026

On the Statistical Benefits of Curriculum Learning

arXiv 2111.07126 ↗First posted 13 Nov 2021stat.ML cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Curriculum learning (CL) is a commonly used machine learning training strategy. However, we still lack a clear theoretical understanding of CL's benefits. In this paper, we study the benefits of CL in the multitask linear regression problem under both structured and unstructured settings. For both settings, we derive the minimax rates for CL with the oracle that provides the optimal curriculum and without the oracle, where the agent has to adaptively learn a good curriculum. Our results reveal that adaptive learning can be fundamentally harder than the oracle learning in the unstructured setting, but it merely introduces a small extra term in the structured setting. To connect theory with practice, we provide justification for a popular empirical method that selects tasks with highest local prediction gain by comparing its guarantees with the minimax rates mentioned above.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 16 of its 32 references that are in the map. First posted on arXiv 13 Nov 2021.

Builds on

16 of 32 references.

References · 16

  1. 2018Deep Visual Domain Adaptation: A Surveycited by 634
  2. 2015An Introduction to Matrix Concentration Inequalitiescited by 391
  3. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 350
  4. 2017Automated Curriculum Learning for Neural Networkscited by 251
  5. 2020Few-Shot Learning via Learning the Representation, Provablycited by 213
  6. 2015The Benefit of Multitask Representation Learningcited by 194
  7. 2020On the Theory of Transfer Learning: The Importance of Task Diversitycited by 186
  8. 2020Deep Visual Domain Adaptationcited by 91
  9. 2020Few-shot Domain Adaptation by Causal Mechanism Transfercited by 47
  10. 2018Theory of Curriculum Learning, with Convex Loss Functionscited by 33
  11. 2018Attention-Guided Curriculum Learning for Weakly Supervised Classification and Localization of Thoracic Diseases on Chest Radiographscited by 32
  12. 2020Minimax Lower Bounds for Transfer Learning with Linear and One-hidden Layer Neural Networkscited by 30
4 more
  1. 2020Adaptive Task Sampling for Meta-Learningcited by 29
  2. 2021Representation Learning Beyond Linear Prediction Functionscited by 21
  3. 2021How to distribute data across tasks for meta-learning?cited by 7
  4. 2020Decision Making Problems with Funnel Structure: A Multi-Task Learning Approach with Application to Email Marketing Campaignscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 6; 2025: 0; 2026: 1.

10 papers.

  1. 2024Disentangling and Mitigating the Impact of Task Similarity for Continual Learningcited by 19
  2. 2022Provable and Efficient Continual Representation Learningcited by 14
  3. 2024Diversify and Conquer: Diversity-Centric Data Selection with Iterative Refinementcited by 13
  4. 2023Provable Pathways: Learning Multiple Tasks over Multiple Pathscited by 3
  5. 2023Curricular and Cyclical Loss for Time Series Learning Strategycited by 1
  6. 2024Statistical curriculum learning: An elimination algorithm achieving an oracle riskcited by 1
  7. 2024Sample Efficient Myopic Exploration Through Multitask Reinforcement Learning with Diverse Taskscited by 1
  8. 2024Solving a Real-World Optimization Problem Using Proximal Policy Optimization with Curriculum Learning and Reward Engineeringcited by 1
  9. 2024Improving the Convergence Rates of Forward Gradient Descent with Repeated Samplingcited by 1
  10. 2026Interactive LLM-assisted Curriculum Learning for Multi-Task Evolutionary Policy Searchcited by 0