mlsanityData to 7 Oct 2026

Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learning

arXiv 2203.06359 ↗First posted 12 Mar 2022cs.CVcited by 118

Not on the map. Why?

Abstract (arXiv)

Non-exemplar class-incremental learning is to recognize both the old and new classes when old class samples cannot be saved. It is a challenging task since representation optimization and feature retention can only be achieved under supervision from new classes. To address this problem, we propose a novel self-sustaining representation expansion scheme. Our scheme consists of a structure reorganization strategy that fuses main-branch expansion and side-branch updating to maintain the old features, and a main-branch distillation scheme to transfer the invariant knowledge. Furthermore, a prototype selection mechanism is proposed to enhance the discrimination between the old and new classes by selectively incorporating new samples into the distillation process. Extensive experiments on three benchmarks demonstrate significant incremental performance, outperforming the state-of-the-art methods by a margin of 3%, 3% and 6%, respectively.

Key facts. Cited by 118 papers in the map (data through 7 Oct 2026). Builds on 36 of its 52 references that are in the map. First posted on arXiv 12 Mar 2022.

Builds on

36 of 52 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  6. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  7. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  8. 2016Progressive Neural Networkscited by 1,332
  9. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  10. 2019Relational Knowledge Distillationcited by 873
  11. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
  12. 2019Large Scale Incremental Learningcited by 727
24 more
  1. 2017Deep Mutual Learningcited by 621
  2. 2019Three scenarios for continual learningcited by 610
  3. 2018End-to-End Incremental Learningcited by 596
  4. 2017Learning multiple visual domains with residual adapterscited by 553
  5. 2018Born Again Neural Networkscited by 539
  6. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 520
  7. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
  8. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  9. 2020Semantic Drift Compensation for Class-Incremental Learningcited by 241
  10. 2020Mnemonics Training: Multi-Class Incremental Learning without Forgettingcited by 220
  11. 2021ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Biascited by 216
  12. 2019Compacting, Picking and Growing for Unforgetting Continual Learningcited by 207
  13. 2019ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blockscited by 178
  14. 2018Efficient parametrization of multi-domain deep neural networkscited by 168
  15. 2020Adaptive Aggregation Networks for Class-Incremental Learningcited by 149
  16. 2021Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learningcited by 130
  17. 2021Distilling Causal Effect of Data in Class-Incremental Learningcited by 117
  18. 2022ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyondcited by 117
  19. 2021Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learningcited by 95
  20. 2021On Learning the Geodesic Path for Incremental Learningcited by 85
  21. 2021Group-aware Label Transfer for Domain Adaptive Person Re-identificationcited by 58
  22. 2019Learning Deep Bilinear Transformation for Fine-grained Image Representationcited by 54
  23. 2018ExpandNets: Linear Over-parameterization to Train Compact Convolutional Networkscited by 53
  24. 2021Revisiting Dynamic Convolution via Matrix Decompositioncited by 35

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 25; 2024: 46; 2025: 30; 2026: 13.

118 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2023Class-Incremental Learning: A Surveycited by 209
  3. 2022FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learningcited by 120
  4. 2023FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learningcited by 86
  5. 2024MagMax: Leveraging Model Merging for Seamless Continual Learningcited by 67
  6. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 66
  7. 2024Class-Incremental Learning with CLIP: Adaptive Representation Adjustment and Parameter Fusioncited by 47
  8. 2023When Prompt-based Incremental Learning Does Not Meet Strong Pretrainingcited by 41
  9. 2024Divide and not forget: Ensemble of selectively trained experts in Continual Learningcited by 38
  10. 2024Elastic Feature Consolidation for Cold Start Exemplar-Free Incremental Learningcited by 37
  11. 2024Make Continual Learning Stronger via C-Flatcited by 35
  12. 2024Resurrecting Old Classes with New Data for Exemplar-Free Continual Learningcited by 33
38 more
  1. 2024Exemplar-free Continual Representation Learning via Learnable Drift Compensationcited by 30
  2. 2024Beyond Prompt Learning: Continual Adapter for Efficient Rehearsal-Free Continual Learningcited by 29
  3. 2023Towards Continual Egocentric Activity Recognition: A Multi-modal Egocentric Activity Dataset for Continual Learningcited by 26
  4. 2023Tangent Model Composition for Ensembling and Continual Fine-tuningcited by 25
  5. 2024DiffClass: Diffusion-Based Class Incremental Learningcited by 23
  6. 2023Continual Segment: Towards a Single, Unified and Accessible Continual Segmentation Model of 143 Whole-body Organs in CT Scanscited by 21
  7. 2024PILoRA: Prototype Guided Incremental LoRA for Federated Class-Incremental Learningcited by 19
  8. 2023Class Incremental Learning with Self-Supervised Pre-Training and Prototype Learningcited by 18
  9. 2025LoRA Subtraction for Drift-Resistant Space in Exemplar-Free Continual Learningcited by 16
  10. 2022Resolving Task Confusion in Dynamic Expansion Architectures for Class Incremental Learningcited by 15
  11. 2023Revisiting Distillation for Continual Learning on Visual Question Localized-Answering in Robotic Surgerycited by 15
  12. 2024Task-recency bias strikes back: Adapting covariances in Exemplar-Free Class Incremental Learningcited by 14
  13. 2023Non-Exemplar Online Class-incremental Continual Learning via Dual-prototype Self-augment and Refinementcited by 13
  14. 2024Continual All-in-One Adverse Weather Removal with Knowledge Replay on a Unified Network Structurecited by 13
  15. 2024Bayesian Learning-driven Prototypical Contrastive Loss for Class-Incremental Learningcited by 13
  16. 2024Rethinking Few-shot Class-incremental Learning: Learning from Yourselfcited by 13
  17. 2023NAPA-VQ: Neighborhood Aware Prototype Augmentation with Vector Quantization for Continual Learningcited by 12
  18. 2024F-OAL: Forward-only Online Analytic Learning with Fast Training and Low Memory Footprint in Class Incremental Learningcited by 12
  19. 2024Learning More Generalized Experts by Merging Experts in Mixture-of-Expertscited by 12
  20. 2024Dynamic Integration of Task-Specific Adapters for Class Incremental Learningcited by 11
  21. 2025ProtoDepth: Unsupervised Continual Depth Completion with Prototypescited by 11
  22. 2024Multi-Granularity Class Prototype Topology Distillation for Class-Incremental Source-Free Unsupervised Domain Adaptationcited by 10
  23. 2022Learning with Style: Continual Semantic Segmentation Across Tasks and Domainscited by 9
  24. 2024Embracing Massive Medical Datacited by 9
  25. 2022Task-Adaptive Saliency Guidance for Exemplar-free Class Incremental Learningcited by 8
  26. 2023An Analysis of Initial Training Strategies for Exemplar-Free Class-Incremental Learningcited by 7
  27. 2024PASS++: A Dual Bias Reduction Framework for Non-Exemplar Class-Incremental Learningcited by 7
  28. 2024Continual Learning with Neuromorphic Computing: Foundations, Methods, and Emerging Applicationscited by 7
  29. 2025Order-Robust Class Incremental Learning: Graph-Driven Dynamic Similarity Groupingcited by 7
  30. 2025Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Class-Incremental Learningcited by 7
  31. 2025AnaCP: Toward Upper-Bound Continual Learning via Analytic Contrastive Projectioncited by 7
  32. 2023Non-exemplar Class-incremental Learning by Random Auxiliary Classes Augmentation and Mixed Featurescited by 6
  33. 2024REAL: Representation Enhanced Analytic Learning for Exemplar-free Class-incremental Learningcited by 6
  34. 2024An Empirical Analysis of Forgetting in Pre-trained Models with Incremental Low-Rank Updatescited by 6
  35. 2024Relation-Guided Adversarial Learning for Data-free Knowledge Transfercited by 6
  36. 2023Category Adaptation Meets Projected Distillation in Generalized Continual Category Discoverycited by 5
  37. 2024CEAT: Continual Expansion and Absorption Transformer for Non-Exemplar Class-Incremental Learningcited by 5
  38. 2024FeTrIL++: Feature Translation for Exemplar-Free Class-Incremental Learning with Hill-Climbingcited by 5

50 most cited of 118 All 118 (JSON)