mlsanityData to 7 Oct 2026

Statistical Measures For Defining Curriculum Scoring Function

arXiv 2103.00147 ↗First posted 27 Feb 2021cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Curriculum learning is a training strategy that sorts the training examples by some measure of their difficulty and gradually exposes them to the learner to improve the network performance. Motivated by our insights from implicit curriculum ordering, we first introduce a simple curriculum learning strategy that uses statistical measures such as standard deviation and entropy values to score the difficulty of data points for real image classification tasks. We empirically show its improvements in performance with convolutional and fully-connected neural networks on multiple real image datasets. We also propose and study the performance of a dynamic curriculum learning algorithm. Our dynamic curriculum algorithm tries to reduce the distance between the network weight and an optimal weight at any training step by greedily sampling examples with gradients that are directed towards the optimal weight. Further, we use our algorithms to discuss why curriculum learning is helpful.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 26 of its 43 references that are in the map. First posted on arXiv 27 Feb 2021.

Builds on

26 of 43 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  5. 2019On the Convergence of Adam and Beyondcited by 834
  6. 2018Not All Samples Are Created Equal: Deep Learning with Importance Samplingcited by 323
  7. 2019On The Power of Curriculum Learning in Training Deep Networkscited by 285
  8. 2017Automated Curriculum Learning for Neural Networkscited by 251
  9. 2017The Marginal Value of Adaptive Gradient Methods in Machine Learningcited by 248
  10. 2017Teacher-Student Curriculum Learningcited by 197
  11. 2017Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samplescited by 147
  12. 2018CurriculumNet: Weakly Supervised Learning from Large-Scale Web Imagescited by 126
14 more
  1. 2015Online Batch Selection for Faster Training of Neural Networkscited by 121
  2. 2017Improving Generalization Performance by Switching from Adam to SGDcited by 121
  3. 2018Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networkscited by 113
  4. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 113
  5. 2020When Do Curricula Work?cited by 105
  6. 2014Stochastic Optimization with Importance Samplingcited by 91
  7. 2020Automatic Curriculum Learning through Value Disagreementcited by 86
  8. 2017Curriculum Learning and Minibatch Bucketing in Neural Machine Translationcited by 63
  9. 2020Norm-Based Curriculum Learning for Neural Machine Translationcited by 60
  10. 2019Curriculum Learning for Domain Adaptation in Neural Machine Translationcited by 59
  11. 2018An Empirical Exploration of Curriculum Learning for Neural Machine Translationcited by 58
  12. 2019Let's Agree to Agree: Neural Networks Share Classification Order on Real Datasetscited by 55
  13. 2020Curriculum Audiovisual Learningcited by 14
  14. 2016Enhanced Directional Smoothing Algorithm for Edge-Preserving Smoothing of Synthetic-Aperture Radar Imagescited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1.

1 paper.

  1. 2022Was that so hard? Estimating human classification difficultycited by 3