mlsanityData to 7 Oct 2026

DER: Dynamically Expandable Representation for Class Incremental Learning

arXiv 2103.16788 ↗First posted 31 Mar 2021cs.CV cs.LGcited by 355

Not on the map. Why?

Abstract (arXiv)

We address the problem of class incremental learning, which is a core step towards achieving adaptive vision intelligence. In particular, we consider the task setting of incremental learning with limited memory and aim to achieve better stability-plasticity trade-off. To this end, we propose a novel two-stage learning approach that utilizes a dynamically expandable representation for more effective incremental concept modeling. Specifically, at each incremental step, we freeze the previously learned representation and augment it with additional feature dimensions from a new learnable feature extractor. This enables us to integrate new visual concepts with retaining learned knowledge. We dynamically expand the representation according to the complexity of novel concepts by introducing a channel-level mask-based pruning strategy. Moreover, we introduce an auxiliary loss to encourage the model to learn diverse and discriminate features for novel concepts. We conduct extensive experiments on the three class incremental learning benchmarks and our method consistently outperforms other methods with a large margin.

Key facts. Cited by 355 papers in the map (data through 7 Oct 2026). Builds on 22 of its 40 references that are in the map. First posted on arXiv 31 Mar 2021.

Builds on

22 of 40 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  5. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  6. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  7. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  8. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
  9. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
  10. 2019Large Scale Incremental Learningcited by 727
  11. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  12. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
10 more
  1. 2018End-to-End Incremental Learningcited by 596
  2. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 362
  3. 2019Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgettingcited by 302
  4. 2019Maintaining Discrimination and Fairness in Class Incremental Learningcited by 281
  5. 2017Overcoming Catastrophic Forgetting by Incremental Moment Matchingcited by 243
  6. 2020Semantic Drift Compensation for Class-Incremental Learningcited by 241
  7. 2019Compacting, Picking and Growing for Unforgetting Continual Learningcited by 207
  8. 2019Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learningcited by 181
  9. 2020Conditional Channel Gated Networks for Task-Aware Continual Learningcited by 115
  10. 2018Heated-Up Softmax Embeddingcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 59; 2023: 96; 2024: 93; 2025: 63; 2026: 29.

355 papers.

Sort:

By area:

  1. 2021Learning to Prompt for Continual Learningcited by 938
  2. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  3. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 635
  4. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  5. 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
  6. 2022FOSTER: Feature Boosting and Compression for Class-Incremental Learningcited by 228
  7. 2024Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapterscited by 199
  8. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 183
  9. 2022A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learningcited by 152
  10. 2022Federated Class-Incremental Learningcited by 126
  11. 2022Forward Compatible Few-Shot Class-Incremental Learningcited by 125
  12. 2022Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learningcited by 118
38 more
  1. 2022Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidationcited by 113
  2. 2023Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Modelscited by 112
  3. 2022Few-Shot Class-Incremental Learning by Sampling Multi-Phase Taskscited by 87
  4. 2023FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learningcited by 86
  5. 2021PyCIL: A Python Toolbox for Class-Incremental Learningcited by 83
  6. 2023Learning without Forgetting for Vision-Language Modelscited by 79
  7. 2022A Theoretical Study on Solving Continual Learningcited by 78
  8. 2021AFEC: Active Forgetting of Negative Transfer in Continual Learningcited by 73
  9. 2024Continual Learning and Catastrophic Forgettingcited by 73
  10. 2022Representation Compensation Networks for Continual Semantic Segmentationcited by 72
  11. 2024MagMax: Leveraging Model Merging for Seamless Continual Learningcited by 67
  12. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 66
  13. 2022CLIP model is an Efficient Continual Learnercited by 64
  14. 2023PILOT: A Pre-Trained Model-Based Continual Learning Toolboxcited by 64
  15. 2022Don't Stop Learning: Towards Continual Learning for the CLIP Modelcited by 61
  16. 2023Computationally Budgeted Continual Learning: What Does Matter?cited by 61
  17. 2022Class-Incremental Learning with Strong Pre-trained Modelscited by 60
  18. 2022ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protectioncited by 56
  19. 2021Recent Advances of Continual Learning in Computer Vision: An Overviewcited by 55
  20. 2022SparCL: Sparse Continual Learning on the Edgecited by 55
  21. 2023Learning with Fantasy: Semantic-Aware Virtual Contrastive Constraint for Few-Shot Class-Incremental Learningcited by 55
  22. 2023On the Stability-Plasticity Dilemma of Class-Incremental Learningcited by 55
  23. 2023No One Left Behind: Real-World Federated Class-Incremental Learningcited by 53
  24. 2024Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learningcited by 53
  25. 2024Consistent Prompting for Rehearsal-Free Continual Learningcited by 52
  26. 2023Class-Incremental Exemplar Compression for Class-Incremental Learningcited by 50
  27. 2023Introducing Language Guidance in Prompt-based Continual Learningcited by 49
  28. 2023Continual Instruction Tuning for Large Multimodal Modelscited by 49
  29. 2024Class-Incremental Learning with CLIP: Adaptive Representation Adjustment and Parameter Fusioncited by 47
  30. 2023First Session Adaptation: A Strong Replay-Free Baseline for Class-Incremental Learningcited by 46
  31. 2023AttriCLIP: A Non-Incremental Learner for Incremental Knowledge Learningcited by 46
  32. 2023Create Your World: Lifelong Text-to-Image Diffusioncited by 46
  33. 2023Dense Network Expansion for Class Incremental Learningcited by 45
  34. 2022Isolation and Impartial Aggregation: A Paradigm of Incremental Learning without Interferencecited by 44
  35. 2022Memory Efficient Continual Learning with Transformerscited by 43
  36. 2022CoSCL: Cooperation of Small Continual Learners is Stronger than a Big Onecited by 43
  37. 2023Achieving a Better Stability-Plasticity Trade-off via Auxiliary Networks in Continual Learningcited by 41
  38. 2023When Prompt-based Incremental Learning Does Not Meet Strong Pretrainingcited by 41

50 most cited of 355 All 355 (JSON)