mlsanityData to 7 Oct 2026

Curriculum Learning: A Survey

arXiv 2101.10382 ↗First posted 25 Jan 2021cs.LG cs.CL cs.CVcited by 257

Not on the map. Why?

Abstract (arXiv)

Training machine learning models in a meaningful order, from the easy samples to the hard ones, using curriculum learning can provide performance improvements over the standard training approach based on random data shuffling, without any additional computational costs. Curriculum learning strategies have been successfully employed in all areas of machine learning, in a wide range of tasks. However, the necessity of finding a way to rank the samples from easy to hard, as well as the right pacing function for introducing more difficult data can limit the usage of the curriculum approaches. In this survey, we show how these limits have been tackled in the literature, and we present different curriculum learning instantiations for various tasks in machine learning. We construct a multi-perspective taxonomy of curriculum learning approaches by hand, considering various classification criteria. We further build a hierarchical tree of curriculum learning methods using an agglomerative clustering algorithm, linking the discovered clusters with our taxonomy. At the end, we provide some interesting directions for future work.

Key facts. Cited by 257 papers in the map (data through 7 Oct 2026). Builds on 107 of its 212 references that are in the map. First posted on arXiv 25 Jan 2021.

Builds on

107 of 212 references.

References · 107

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  8. 2020End-to-End Object Detection with Transformerscited by 7,384
  9. 2014Going Deeper with Convolutionscited by 4,231
  10. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  11. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  12. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
38 more
  1. 2021Transformers in Vision: A Surveycited by 1,009
  2. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  3. 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
  4. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
  5. 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
  6. 2015Deep Speech 2: End-to-End Speech Recognition in English and Mandarincited by 449
  7. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 350
  8. 2020CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognitioncited by 320
  9. 2019On The Power of Curriculum Learning in Training Deep Networkscited by 285
  10. 2017Automated Curriculum Learning for Neural Networkscited by 251
  11. 2019Competence-based Curriculum Learning for Neural Machine Translationcited by 201
  12. 2017Teacher-Student Curriculum Learningcited by 197
  13. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 185
  14. 2018Speaker Recognition from Raw Waveform with SincNetcited by 166
  15. 2017Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samplescited by 147
  16. 2020Anomaly Detection in Video via Self-Supervised and Multi-Task Learningcited by 147
  17. 2015Character-Aware Neural Language Modelscited by 129
  18. 2018CurriculumNet: Weakly Supervised Learning from Large-Scale Web Imagescited by 126
  19. 2018Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networkscited by 113
  20. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 113
  21. 2014Learning to Executecited by 105
  22. 2015Webly Supervised Learning of Convolutional Networkscited by 95
  23. 2019Dynamic Curriculum Learning for Imbalanced Data Classificationcited by 93
  24. 2020Automatic Curriculum Learning through Value Disagreementcited by 86
  25. 2020Multi-Task Curriculum Framework for Open-Set Semi-Supervised Learningcited by 82
  26. 2020PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learningcited by 79
  27. 2020Safe Reinforcement Learning via Curriculum Inductioncited by 73
  28. 2014Curriculum Learning of Multiple Taskscited by 65
  29. 2018Efficient and Accurate MRI Super-Resolution using a Generative Adversarial Network and 3D Multi-Level Densely Connected Networkcited by 64
  30. 2017Curriculum Learning and Minibatch Bucketing in Neural Machine Translationcited by 63
  31. 2020Curriculum DeepSDFcited by 62
  32. 2020Norm-Based Curriculum Learning for Neural Machine Translationcited by 60
  33. 2019Curriculum Learning for Domain Adaptation in Neural Machine Translationcited by 59
  34. 2019Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narrativescited by 59
  35. 2018An Empirical Exploration of Curriculum Learning for Neural Machine Translationcited by 58
  36. 2020Curriculum Pre-training for End-to-End Speech Translationcited by 57
  37. 2020Curriculum Manager for Source Selection in Multi-Source Domain Adaptationcited by 55
  38. 2017How hard can it be? Estimating the difficulty of visual search in an imagecited by 52

50 most cited of 107 All 107 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 25; 2023: 48; 2024: 49; 2025: 75; 2026: 45.

257 papers.

Sort:

By area:

  1. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  2. 2021BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Renderingcited by 228
  3. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 121
  4. 2023Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Modelscited by 120
  5. 2021Mutual Consistency Learning for Semi-supervised Medical Image Segmentationcited by 107
  6. 2023LLaFS: When Large Language Models Meet Few-Shot Segmentationcited by 48
  7. 2022EfficientTrain: Exploring Generalized Curriculum Learning for Training Visual Backbonescited by 42
  8. 2024Curriculum Direct Preference Optimization for Diffusion and Consistency Modelscited by 36
  9. 2023CLIP-VG: Self-paced Curriculum Adapting of CLIP for Visual Groundingcited by 35
  10. 2024How Far Can Transformers Reason? The Globality Barrier and Inductive Scratchpadcited by 35
  11. 2025VCRL: Variance-based Curriculum Reinforcement Learning for Large Language Modelscited by 35
  12. 2021Curriculum learning for language modelingcited by 32
38 more
  1. 2025VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understandingcited by 32
  2. 2024Fake It till You Make It: Curricular Dynamic Forgery Augmentations towards General Deepfake Detectioncited by 30
  3. 2023Semantic Image Segmentation: Two Decades of Researchcited by 26
  4. 2023On The Impact of Machine Learning Randomness on Group Fairnesscited by 26
  5. 2021Hybrid Curriculum Learning for Emotion Recognition in Conversationcited by 25
  6. 2022Incremental Spatial and Spectral Learning of Neural Operators for Solving Large-Scale PDEscited by 25
  7. 2023Provable Advantage of Curriculum Learning on Parity Targets with Mixed Inputscited by 22
  8. 2024A Practitioner's Guide to Continual Multimodal Pretrainingcited by 22
  9. 2021Deep Anomaly Discovery From Unlabeled Videos via Normality Advantage and Self-Paced Refinementcited by 21
  10. 2023CL-MAE: Curriculum-Learned Masked Autoencoderscited by 21
  11. 2023Improving Plasticity in Online Continual Learning via Collaborative Learningcited by 20
  12. 2023A Mathematical Model for Curriculum Learning for Paritiescited by 19
  13. 2024CurricuLLM: Automatic Task Curricula Design for Learning Complex Robot Skills using Large Language Modelscited by 19
  14. 2025Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesiscited by 19
  15. 2022GraB: Finding Provably Better Data Permutations than Random Reshufflingcited by 18
  16. 2023Multiview Aerial Visual Recognition (MAVREC): Can Multi-view Improve Aerial Visual Perception?cited by 17
  17. 2023Camera-Driven Representation Learning for Unsupervised Domain Adaptive Person Re-identificationcited by 16
  18. 2024EfficientTrain++: Generalized Curriculum Learning for Efficient Visual Backbone Trainingcited by 16
  19. 2025Chiron-o1: Igniting Multimodal Large Language Models towards Generalizable Medical Reasoning via Mentor-Intern Collaborative Searchcited by 16
  20. 2023Curriculum Graph Machine Learning: A Surveycited by 15
  21. 2025DC-Scene: Data-Centric Learning for 3D Scene Understandingcited by 15
  22. 2022Contextual Similarity is More Valuable than Character Similarity: An Empirical Study for Chinese Spell Checkingcited by 14
  23. 2021Improving Self-supervised Learning with Hardness-aware Dynamic Curriculum Learning: An Application to Digital Pathologycited by 13
  24. 2024Intent-based Prompt Calibration: Enhancing prompt optimization with synthetic boundary casescited by 13
  25. 2024Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferencescited by 13
  26. 2024EntAugment: Entropy-Driven Adaptive Data Augmentation Framework for Image Classificationcited by 13
  27. 2023Curricular Object Manipulation in LiDAR-based Object Detectioncited by 12
  28. 2025Bottom-up Domain-specific Superintelligence: A Reliable Knowledge Graph is What We Needcited by 12
  29. 2021Curriculum Learning for Safe Mapless Navigationcited by 11
  30. 2022Generalization in Neural Networks: A Broad Surveycited by 11
  31. 2023MindLLM: Pre-training Lightweight Large Language Model from Scratch, Evaluations and Domain Applicationscited by 11
  32. 2024CaT: Constraints as Terminations for Legged Locomotion Reinforcement Learningcited by 11
  33. 2025STPro: Spatial and Temporal Progressive Learning for Weakly Supervised Spatio-Temporal Groundingcited by 11
  34. 2025IRGPT: Understanding Real-world Infrared Image with Bi-cross-modal Curriculum on Large-scale Benchmarkcited by 11
  35. 2025RecGPT Technical Reportcited by 11
  36. 2025Foundation Models for Bioacoustics -- a Comparative Reviewcited by 11
  37. 2021Unsupervised Medical Image Alignment with Curriculum Learningcited by 10
  38. 2021Self-paced ensemble learning for speech and audio classificationcited by 10

50 most cited of 257 All 257 (JSON)