mlsanityData to 7 Oct 2026

End-to-End Incremental Learning

arXiv 1807.09536 ↗First posted 25 Jul 2018cs.CVcited by 596

Not on the map. Why?

Abstract (arXiv)

Although deep learning approaches have stood out in recent years due to their state-of-the-art results, they continue to suffer from catastrophic forgetting, a dramatic decrease in overall performance when training with new classes added incrementally. This is due to current neural network architectures requiring the entire dataset, consisting of all the samples from the old as well as the new classes, to update the model -a requirement that becomes easily unsustainable as the number of classes grows. We address this issue with our approach to learn deep neural networks incrementally, using new data and only a small exemplar set corresponding to samples from the old classes. This is based on a loss composed of a distillation measure to retain the knowledge acquired from the old classes, and a cross-entropy loss to learn the new classes. Our incremental training is achieved while keeping the entire framework end-to-end, i.e., learning the data representation and the classifier jointly, unlike recent methods with no such guarantees. We evaluate our method extensively on the CIFAR-100 and ImageNet (ILSVRC 2012) image classification datasets, and show state-of-the-art performance.

Key facts. Cited by 596 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 110; 2022: 143; 2023: 138; 2024: 118; 2025: 55; 2026: 32.

596 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 446
  3. 2021Dataset Condensation with Distribution Matchingcited by 362
  4. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  5. 2021Towards Open World Object Detectioncited by 297
  6. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  7. 2021Dataset Condensation with Differentiable Siamese Augmentationcited by 290
  8. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  9. 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
  10. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  11. 2022FOSTER: Feature Boosting and Compression for Class-Incremental Learningcited by 228
  12. 2023Class-Incremental Learning: A Surveycited by 209
38 more
  1. 2024Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapterscited by 199
  2. 2023Robust Test-Time Adaptation in Dynamic Scenarioscited by 185
  3. 2021Few-Shot Incremental Learning with Continually Evolved Classifierscited by 184
  4. 2022Scaling Up Dataset Distillation to ImageNet-1K with Constant Memorycited by 180
  5. 2022Dataset Distillation using Neural Feature Regressioncited by 177
  6. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 162
  7. 2022A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learningcited by 152
  8. 2021Graph Condensation for Graph Neural Networkscited by 150
  9. 2021Self-Supervised Models are Continual Learnerscited by 149
  10. 2022Dataset Pruning: Reducing Training Data by Examining Generalization Influencecited by 132
  11. 2021Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learningcited by 130
  12. 2022Forward Compatible Few-Shot Class-Incremental Learningcited by 125
  13. 2023Improved Distribution Matching for Dataset Condensationcited by 124
  14. 2022Class-Incremental Continual Learning into the eXtended DER-versecited by 121
  15. 2022FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learningcited by 120
  16. 2021Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minimacited by 118
  17. 2022Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learningcited by 118
  18. 2021Distilling Causal Effect of Data in Class-Incremental Learningcited by 117
  19. 2022Dataset Distillation via Factorizationcited by 115
  20. 2021Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learningcited by 114
  21. 2023A Comprehensive Survey of Dataset Distillationcited by 114
  22. 2022Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidationcited by 113
  23. 2023Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learningcited by 112
  24. 2023Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Modelscited by 112
  25. 2022Efficient Dataset Distillation Using Random Feature Approximationcited by 111
  26. 2022Condensing Graphs via One-Step Gradient Matchingcited by 100
  27. 2021Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representationscited by 96
  28. 2021Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learningcited by 95
  29. 2022Constrained Few-shot Class-incremental Learningcited by 93
  30. 2022Few-Shot Class-Incremental Learning by Sampling Multi-Phase Taskscited by 87
  31. 2023FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learningcited by 86
  32. 2021On Learning the Geodesic Path for Incremental Learningcited by 85
  33. 2022Few-Shot Class-Incremental Learning from an Open-Set Perspectivecited by 83
  34. 2023DataDAM: Efficient Dataset Distillation with Attention Matchingcited by 83
  35. 2022A Theoretical Study on Solving Continual Learningcited by 78
  36. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
  37. 2021Rehearsal revealed: The limits and merits of revisiting samples in continual learningcited by 76
  38. 2021Replay in Deep Learning: Current Approaches and Missing Biological Elementscited by 71

50 most cited of 596 All 596 (JSON)