mlsanityData to 7 Oct 2026

FOSTER: Feature Boosting and Compression for Class-Incremental Learning

arXiv 2204.04662 ↗First posted 10 Apr 2022cs.CV cs.LGcited by 228

Not on the map. Why?

Abstract (arXiv)

The ability to learn new concepts continually is necessary in this ever-changing world. However, deep neural networks suffer from catastrophic forgetting when learning new categories. Many works have been proposed to alleviate this phenomenon, whereas most of them either fall into the stability-plasticity dilemma or take too much computation or storage overhead. Inspired by the gradient boosting algorithm to gradually fit the residuals between the target model and the previous ensemble model, we propose a novel two-stage learning paradigm FOSTER, empowering the model to learn new categories adaptively. Specifically, we first dynamically expand new modules to fit the residuals between the target and the output of the original model. Next, we remove redundant parameters and feature dimensions through an effective distillation strategy to maintain the single backbone model. We validate our method FOSTER on CIFAR-100 and ImageNet-100/1000 under different settings. Experimental results show that our method achieves state-of-the-art performance. Code is available at: https://github.com/G-U-N/ECCV22-FOSTER.

Key facts. Cited by 228 papers in the map (data through 7 Oct 2026). Builds on 39 of its 63 references that are in the map. First posted on arXiv 10 Apr 2022.

Builds on

39 of 63 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  4. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  7. 2016Learning without Forgettingcited by 2,465
  8. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  9. 2016Progressive Neural Networkscited by 1,332
  10. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  11. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  12. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
27 more
  1. 2019Large Scale Incremental Learningcited by 727
  2. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  3. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  4. 2018End-to-End Incremental Learningcited by 596
  5. 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 446
  6. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 368
  7. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 362
  8. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  9. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  10. 2019Maintaining Discrimination and Fairness in Class Incremental Learningcited by 281
  11. 2018CatBoost: gradient boosting with categorical features supportcited by 208
  12. 2019Compacting, Picking and Growing for Unforgetting Continual Learningcited by 207
  13. 2022Forward Compatible Few-Shot Class-Incremental Learningcited by 125
  14. 2021Learning Placeholders for Open-Set Recognitioncited by 121
  15. 2020Memory-Efficient Incremental Learning Through Feature Adaptationcited by 107
  16. 2019Continual Learning via Neural Pruningcited by 101
  17. 2022Few-Shot Class-Incremental Learning by Sampling Multi-Phase Taskscited by 87
  18. 2021PyCIL: A Python Toolbox for Class-Incremental Learningcited by 83
  19. 2022Memory Replay with Data Compression for Continual Learningcited by 80
  20. 2021AFEC: Active Forgetting of Negative Transfer in Continual Learningcited by 73
  21. 2018Generative Models from the perspective of Continual Learningcited by 69
  22. 2021ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learningcited by 68
  23. 2015Bayesian Dark Knowledgecited by 45
  24. 2021Co-Transport for Class-Incremental Learningcited by 38
  25. 2020Incremental Learning via Rate Reductioncited by 31
  26. 2021Balanced Knowledge Distillation for Long-tailed Learningcited by 23
  27. 2021Preserving Earlier Knowledge in Continual Learning with the Help of All Previous Feature Extractorscited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 48; 2024: 78; 2025: 62; 2026: 36.

228 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2023Class-Incremental Learning: A Surveycited by 209
  3. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 183
  4. 2022A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learningcited by 152
  5. 2023Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoisingcited by 135
  6. 2024Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learningcited by 115
  7. 2023FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learningcited by 86
  8. 2023Learning without Forgetting for Vision-Language Modelscited by 79
  9. 2024MagMax: Leveraging Model Merging for Seamless Continual Learningcited by 67
  10. 2023TARGET: Federated Class-Continual Learning via Exemplar-Free Distillationcited by 64
  11. 2023PILOT: A Pre-Trained Model-Based Continual Learning Toolboxcited by 64
  12. 2023Continual Detection Transformer for Incremental Object Detectioncited by 59
38 more
  1. 2024Consistent Prompting for Rehearsal-Free Continual Learningcited by 52
  2. 2023Class-Incremental Exemplar Compression for Class-Incremental Learningcited by 50
  3. 2023A Survey on Few-Shot Class-Incremental Learningcited by 48
  4. 2024Class-Incremental Learning with CLIP: Adaptive Representation Adjustment and Parameter Fusioncited by 47
  5. 2023Dense Network Expansion for Class Incremental Learningcited by 45
  6. 2023Few-Shot Class-Incremental Learning via Training-Free Prototype Calibrationcited by 44
  7. 2023When Prompt-based Incremental Learning Does Not Meet Strong Pretrainingcited by 41
  8. 2024Divide and not forget: Ensemble of selectively trained experts in Continual Learningcited by 38
  9. 2024Gradient Reweighting: Towards Imbalanced Class-Incremental Learningcited by 37
  10. 2023Online Hyperparameter Optimization for Class-Incremental Learningcited by 36
  11. 2023Federated Incremental Semantic Segmentationcited by 36
  12. 2023Online Class Incremental Learning on Stochastic Blurry Task Boundary via Mask and Visual Prompt Tuningcited by 35
  13. 2024Make Continual Learning Stronger via C-Flatcited by 35
  14. 2023Audio-Visual Class-Incremental Learningcited by 33
  15. 2024Resurrecting Old Classes with New Data for Exemplar-Free Continual Learningcited by 33
  16. 2023Class-Incremental Learning using Diffusion Model for Distillation and Replaycited by 31
  17. 2024Beyond Prompt Learning: Continual Adapter for Efficient Rehearsal-Free Continual Learningcited by 29
  18. 2023Towards Continual Egocentric Activity Recognition: A Multi-modal Egocentric Activity Dataset for Continual Learningcited by 26
  19. 2023Dynamic Residual Classifier for Class Incremental Learningcited by 26
  20. 2024Online-LoRA: Task-free Online Continual Learning via Low Rank Adaptationcited by 26
  21. 2025CL-LoRA: Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learningcited by 26
  22. 2023SIESTA: Efficient Online Continual Learning with Sleepcited by 24
  23. 2024DiffClass: Diffusion-Based Class Incremental Learningcited by 23
  24. 2025Synthetic Data is an Elegant GIFT for Continual Vision-Language Modelscited by 21
  25. 2025Integrating Task-Specific and Universal Adapters for Pre-Trained Model-based Class-Incremental Learningcited by 19
  26. 2023Rethinking Momentum Knowledge Distillation in Online Continual Learningcited by 18
  27. 2024Adapter Merging with Centroid Prototype Mapping for Scalable Class-Incremental Learningcited by 18
  28. 2022A soft nearest-neighbor framework for continual semi-supervised learningcited by 17
  29. 2023Class Incremental Learning via Likelihood Ratio Based Task Predictioncited by 17
  30. 2024Revisiting Neural Networks for Continual Learning: An Architectural Perspectivecited by 17
  31. 2022Continual Prune-and-Select: Class-incremental learning with specialized subnetworkscited by 16
  32. 2024Dual Consolidation for Pre-Trained Model-Based Domain-Incremental Learningcited by 16
  33. 2025LoRA Subtraction for Drift-Resistant Space in Exemplar-Free Continual Learningcited by 16
  34. 2023Heterogeneous Forgetting Compensation for Class-Incremental Learningcited by 15
  35. 2024Few-Shot Class Incremental Learning with Attention-Aware Self-Adaptive Promptcited by 15
  36. 2024CoLeCLIP: Open-Domain Continual Learning via Joint Task Prompt and Vocabulary Learningcited by 15
  37. 2025Task-Agnostic Guided Feature Expansion for Class-Incremental Learningcited by 15
  38. 2023Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variantscited by 14

50 most cited of 228 All 228 (JSON)