mlsanityData to 7 Oct 2026

Large Scale Incremental Learning

arXiv 1905.13260 ↗First posted 30 May 2019cs.CVcited by 727

Not on the map. Why?

Abstract (arXiv)

Modern machine learning suffers from catastrophic forgetting when learning new classes incrementally. The performance dramatically degrades due to the missing data of old classes. Incremental learning methods have been proposed to retain the knowledge acquired from the old classes, by using knowledge distilling and keeping a few exemplars from the old classes. However, these methods struggle to scale up to a large number of classes. We believe this is because of the combination of two factors: (a) the data imbalance between the old and new classes, and (b) the increasing number of visually similar classes. Distinguishing between an increasing number of visually similar classes is particularly challenging, when the training data is unbalanced. We propose a simple and effective method to address this data imbalance issue. We found that the last fully connected layer has a strong bias towards the new classes, and this bias can be corrected by a linear model. With two bias parameters, our method performs remarkably well on two large datasets: ImageNet (1000 classes) and MS-Celeb-1M (10000 classes), outperforming the state-of-the-art algorithms by 11.1% and 13.2% respectively.

Key facts. Cited by 727 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 100; 2022: 154; 2023: 169; 2024: 157; 2025: 106; 2026: 41.

727 papers.

Sort:

By area:

  1. 2021Learning to Prompt for Continual Learningcited by 938
  2. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  3. 2021Domain Generalization: A Surveycited by 760
  4. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 635
  5. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 446
  6. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  7. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  8. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  9. 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
  10. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  11. 2022FOSTER: Feature Boosting and Compression for Class-Incremental Learningcited by 228
  12. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
38 more
  1. 2021New Insights on Reducing Abrupt Representation Change in Online Continual Learningcited by 215
  2. 2023Class-Incremental Learning: A Surveycited by 209
  3. 2024Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapterscited by 199
  4. 2021Few-Shot Incremental Learning with Continually Evolved Classifierscited by 184
  5. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 183
  6. 2023SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Modelcited by 178
  7. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 162
  8. 2022A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learningcited by 152
  9. 2021Self-Supervised Models are Continual Learnerscited by 149
  10. 2021Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learningcited by 140
  11. 2021Training Networks in Null Space of Feature Covariance for Continual Learningcited by 139
  12. 2021Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learningcited by 130
  13. 2022Federated Class-Incremental Learningcited by 126
  14. 2022Forward Compatible Few-Shot Class-Incremental Learningcited by 125
  15. 2022Class-Incremental Continual Learning into the eXtended DER-versecited by 121
  16. 2022FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learningcited by 120
  17. 2022Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learningcited by 118
  18. 2021Distilling Causal Effect of Data in Class-Incremental Learningcited by 117
  19. 2024Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learningcited by 115
  20. 2021Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learningcited by 114
  21. 2022Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidationcited by 113
  22. 2023Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Modelscited by 112
  23. 2023BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learningcited by 112
  24. 2022Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning Systemcited by 110
  25. 2021Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learningcited by 95
  26. 2022Constrained Few-shot Class-incremental Learningcited by 93
  27. 2022Few-Shot Class-Incremental Learning by Sampling Multi-Phase Taskscited by 87
  28. 2023FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learningcited by 86
  29. 2021On Learning the Geodesic Path for Incremental Learningcited by 85
  30. 2021PyCIL: A Python Toolbox for Class-Incremental Learningcited by 83
  31. 2022Memory Replay with Data Compression for Continual Learningcited by 80
  32. 2023Learning without Forgetting for Vision-Language Modelscited by 79
  33. 2022A Theoretical Study on Solving Continual Learningcited by 78
  34. 2022Representation Compensation Networks for Continual Semantic Segmentationcited by 72
  35. 2022Overcoming Catastrophic Forgetting in Incremental Object Detection via Elastic Response Distillationcited by 72
  36. 2021Replay in Deep Learning: Current Approaches and Missing Biological Elementscited by 71
  37. 2021SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learningcited by 71
  38. 2021ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learningcited by 68

50 most cited of 727 All 727 (JSON)