Statistical curriculum learning: An elimination algorithm achieving an oracle risk
Not on the map. Why?
Abstract (arXiv)
We consider a statistical version of curriculum learning (CL) in a parametric prediction setting. The learner is required to estimate a target parameter vector, and can adaptively collect samples from either the target model, or other source models that are similar to the target model, but less noisy. We consider three types of learners, depending on the level of side-information they receive. The first two, referred to as strong/weak-oracle learners, receive high/low degrees of information about the models, and use these to learn. The third, a fully adaptive learner, estimates the target parameter vector without any prior information. In the single source case, we propose an elimination learning method, whose risk matches that of a strong-oracle learner. In the multiple source case, we advocate that the risk of the weak-oracle learner is a realistic benchmark for the risk of adaptive learners. We develop an adaptive multiple elimination-rounds CL algorithm, and characterize instance-dependent conditions for its risk to match that of the weak-oracle learner. We consider instance-dependent minimax lower bounds, and discuss the challenges associated with defining the class of instances for the bound. We derive two minimax lower bounds, and determine the conditions under which the performance weak-oracle learner is minimax optimal.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 21 of its 68 references that are in the map. First posted on arXiv 20 Feb 2024.
Builds on
21 of 68 references.
References · 21
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
- 2019Three scenarios for continual learningcited by 614
- 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 375
- 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
- 2019On The Power of Curriculum Learning in Training Deep Networkscited by 286
- 2016Optimal Best Arm Identification with Fixed Confidencecited by 239
- 2020Few-Shot Learning via Learning the Representation, Provablycited by 214
- 2020On the Theory of Transfer Learning: The Importance of Task Diversitycited by 186
- 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 150
- 2014Best-Arm Identification in Linear Banditscited by 100
- 2023Theory on Forgetting and Generalization of Continual Learningcited by 72
9 more
- 2023Continual Learning in Linear Classification on Separable Datacited by 34
- 2018Theory of Curriculum Learning, with Convex Loss Functionscited by 33
- 2020Minimax Lower Bounds for Transfer Learning with Linear and One-hidden Layer Neural Networkscited by 30
- 2022Analysis of Catastrophic Forgetting for Random Orthogonal Transformation Tasks in the Overparameterized Regimecited by 29
- 2024The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting -- An Analytical Modelcited by 26
- 2023Fixed Design Analysis of Regularization-Based Continual Learningcited by 24
- 2020Active Learning: Problem Settings and Recent Developmentscited by 23
- 2023A Mathematical Model for Curriculum Learning for Paritiescited by 19
- 2021On the Statistical Benefits of Curriculum Learningcited by 10
Built on by
Citing papers per year: 2025: 1.
1 paper.