mlsanityData to 7 Oct 2026

Mnemonics Training: Multi-Class Incremental Learning without Forgetting

arXiv 2002.10211 ↗First posted 24 Feb 2020cs.CV stat.MLcited by 220

Not on the map. Why?

Abstract (arXiv)

Multi-Class Incremental Learning (MCIL) aims to learn new concepts by incrementally updating a model trained on previous concepts. However, there is an inherent trade-off to effectively learning new concepts without catastrophic forgetting of previous ones. To alleviate this issue, it has been proposed to keep around a few examples of the previous concepts but the effectiveness of this approach heavily depends on the representativeness of these examples. This paper proposes a novel and automatic framework we call mnemonics, where we parameterize exemplars and make them optimizable in an end-to-end manner. We train the framework through bilevel optimizations, i.e., model-level and exemplar-level. We conduct extensive experiments on three MCIL benchmarks, CIFAR-100, ImageNet-Subset and ImageNet, and show that using mnemonics exemplars can surpass the state-of-the-art by a large margin. Interestingly and quite intriguingly, the mnemonics exemplars tend to be on the boundaries between different classes.

Key facts. Cited by 220 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 45; 2022: 49; 2023: 53; 2024: 35; 2025: 24; 2026: 14.

220 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2022Prompt-aligned Gradient for Prompt Tuningcited by 339
  3. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  4. 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
  5. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  6. 2023Class-Incremental Learning: A Surveycited by 209
  7. 2023Continual Pre-training of Language Modelscited by 194
  8. 2021Few-Shot Incremental Learning with Continually Evolved Classifierscited by 184
  9. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 183
  10. 2022Dataset Distillation using Neural Feature Regressioncited by 177
  11. 2022A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learningcited by 152
  12. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
38 more
  1. 2021Training Networks in Null Space of Feature Covariance for Continual Learningcited by 139
  2. 2022Federated Class-Incremental Learningcited by 126
  3. 2023Dataset Distillation: A Comprehensive Reviewcited by 126
  4. 2022FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learningcited by 120
  5. 2021Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minimacited by 118
  6. 2022Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learningcited by 118
  7. 2021Distilling Causal Effect of Data in Class-Incremental Learningcited by 117
  8. 2024Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learningcited by 115
  9. 2022Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidationcited by 113
  10. 2021Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learningcited by 95
  11. 2022Constrained Few-shot Class-incremental Learningcited by 93
  12. 2021Achieving Forgetting Prevention and Knowledge Transfer in Continual Learningcited by 91
  13. 2021On Learning the Geodesic Path for Incremental Learningcited by 85
  14. 2022Memory Replay with Data Compression for Continual Learningcited by 80
  15. 2023Learning without Forgetting for Vision-Language Modelscited by 79
  16. 2022A Theoretical Study on Solving Continual Learningcited by 78
  17. 2021Replay in Deep Learning: Current Approaches and Missing Biological Elementscited by 71
  18. 2021ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learningcited by 68
  19. 2021Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inferencecited by 66
  20. 2023RMM: Reinforced Memory Management for Class-Incremental Learningcited by 66
  21. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 66
  22. 2024MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learningcited by 63
  23. 2022Don't Stop Learning: Towards Continual Learning for the CLIP Modelcited by 61
  24. 2023Computationally Budgeted Continual Learning: What Does Matter?cited by 61
  25. 2023Continual Detection Transformer for Incremental Object Detectioncited by 59
  26. 2021Essentials for Class Incremental Learningcited by 57
  27. 2022ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protectioncited by 56
  28. 2021Recent Advances of Continual Learning in Computer Vision: An Overviewcited by 55
  29. 2023Class-Incremental Exemplar Compression for Class-Incremental Learningcited by 50
  30. 2021Mimicking the Oracle: An Initial Phase Decorrelation Approach for Class Incremental Learningcited by 49
  31. 2024Class-Incremental Learning with CLIP: Adaptive Representation Adjustment and Parameter Fusioncited by 47
  32. 2022A Continual Deepfake Detection Benchmark: Dataset, Methods, and Essentialscited by 45
  33. 2023When Prompt-based Incremental Learning Does Not Meet Strong Pretrainingcited by 41
  34. 2023MetaGCD: Learning to Continually Learn in Generalized Category Discoverycited by 40
  35. 2021DualNet: Continual Learning, Fast and Slowcited by 39
  36. 2021Incremental Learning in Semantic Segmentation from Image Labelscited by 39
  37. 2022Novel Class Discovery without Forgettingcited by 39
  38. 2022Energy-based Latent Aligner for Incremental Learningcited by 37

50 most cited of 220 All 220 (JSON)