mlsanityData to 7 Oct 2026

Schedule-Robust Online Continual Learning

arXiv 2210.05561 ↗First posted 11 Oct 2022cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

A continual learning (CL) algorithm learns from a non-stationary data stream. The non-stationarity is modeled by some schedule that determines how data is presented over time. Most current methods make strong assumptions on the schedule and have unpredictable performance when such requirements are not met. A key challenge in CL is thus to design methods robust against arbitrary schedules over the same underlying data, since in real-world scenarios schedules are often unknown and dynamic. In this work, we introduce the notion of schedule-robustness for CL and a novel approach satisfying this desirable property in the challenging online class-incremental setting. We also present a new perspective on CL, as the process of learning a schedule-robust predictor, followed by adapting the predictor using only replay data. Empirically, we demonstrate that our approach outperforms existing methods on CL benchmarks for image classification by a large margin.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 43 of its 54 references that are in the map. First posted on arXiv 11 Oct 2022.

Builds on

43 of 54 references.

References · 43

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  4. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  5. 2016Learning without Forgettingcited by 2,465
  6. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  7. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  8. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  9. 2021Learning to Prompt for Continual Learningcited by 938
  10. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
  11. 2019Large Scale Incremental Learningcited by 727
  12. 2019Gradient based sample selection for online continual learningcited by 568
31 more
  1. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 511
  2. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
  3. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
  4. 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
  5. 2018Meta-learning with differentiable closed-form solverscited by 395
  6. 2019Online Continual Learning with Maximally Interfered Retrievalcited by 376
  7. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  8. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 312
  9. 2019On Tiny Episodic Memories in Continual Learningcited by 299
  10. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  11. 2020Semantic Drift Compensation for Class-Incremental Learningcited by 241
  12. 2018Task-Free Continual Learningcited by 219
  13. 2021New Insights on Reducing Abrupt Representation Change in Online Continual Learningcited by 215
  14. 2019Meta-Learning Representations for Continual Learningcited by 189
  15. 2020Understanding the Role of Training Regimes in Continual Learningcited by 177
  16. 2018Towards Robust Evaluations of Continual Learningcited by 148
  17. 2021An Empirical Investigation of the Role of Pre-training in Lifelong Learningcited by 134
  18. 2020SS-IL: Separated Softmax for Incremental Learningcited by 126
  19. 2019Scalable and Order-robust Continual Learning with Additive Parameter Decompositioncited by 125
  20. 2021Cross-domain Few-shot Learning with Task-specific Adapterscited by 79
  21. 2022Class-Incremental Learning with Strong Pre-trained Modelscited by 60
  22. 2019Few-Shot Learning with Embedded Class Models and Shot-Free Meta Trainingcited by 57
  23. 2021Essentials for Class Incremental Learningcited by 57
  24. 2021Universal Representation Learning from Multiple Domains for Few-shot Classificationcited by 56
  25. 2020La-MAML: Look-ahead Meta Learning for Continual Learningcited by 51
  26. 2022Transfer without Forgettingcited by 44
  27. 2021CLEVA-Compass: A Continual Learning EValuation Assessment Compass to Promote Research Transparency and Comparabilitycited by 28
  28. 2020Defining Benchmarks for Continual Few-Shot Learningcited by 21
  29. 2021Encoders and Ensembles for Task-Free Continual Learningcited by 17
  30. 2021The Role of Global Labels in Few-Shot Classification and How to Infer Themcited by 12
  31. 2022Lessons learned from the NeurIPS 2021 MetaDL challenge: Backbone fine-tuning without episodic meta-learning dominates for few-shot learning image classificationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2024Continual Learning and Catastrophic Forgettingcited by 73
  2. 2024Accelerating Heterogeneous Federated Learning with Closed-form Classifierscited by 10
  3. 2026Training-Free Dynamic Upcycling of Expert Language Modelscited by 1
  4. 2025The Unreasonable Effectiveness of Randomized Representations in Online Continual Graph Learningcited by 0