mlsanityData to 14 Oct 2026

CLOSER: Towards Better Representation Learning for Few-Shot Class-Incremental Learning

arXiv 2410.05627 ↗First posted 8 Oct 2024cs.CV cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Aiming to incrementally learn new classes with only few samples while preserving the knowledge of base (old) classes, few-shot class-incremental learning (FSCIL) faces several challenges, such as overfitting and catastrophic forgetting. Such a challenging problem is often tackled by fixing a feature extractor trained on base classes to reduce the adverse effects of overfitting and forgetting. Under such formulation, our primary focus is representation learning on base classes to tackle the unique challenge of FSCIL: simultaneously achieving the transferability and the discriminability of the learned representation. Building upon the recent efforts for enhancing transferability, such as promoting the spread of features, we find that trying to secure the spread of features within a more confined feature space enables the learned representation to strike a better balance between transferability and discriminability. Thus, in stark contrast to prior beliefs that the inter-class distance should be maximized, we claim that the closer different classes are, the better for FSCIL. The empirical results and analysis from the perspective of information bottleneck theory justify our simple yet seemingly counter-intuitive representation learning method, raising research questions and suggesting alternative research directions. The code is available at https://github.com/JungHunOh/CLOSER_ECCV2024.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 44 of its 57 references that are in the map. First posted on arXiv 8 Oct 2024.

Builds on

44 of 57 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  8. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  9. 2016Matching Networks for One Shot Learningcited by 2,255
  10. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  11. 2018CosFace: Large Margin Cosine Loss for Deep Face Recognitioncited by 971
  12. 2016Deep Variational Information Bottleneckcited by 930
32 more
  1. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  2. 2018AutoAugment: Learning Augmentation Policies from Datacited by 730
  3. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
  4. 2018MINE: Mutual Information Neural Estimationcited by 683
  5. 2017Opening the Black Box of Deep Neural Networks via Informationcited by 503
  6. 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 424
  7. 2014CNN Features off-the-shelf: an Astounding Baseline for Recognitioncited by 316
  8. 2020Few-Shot Class-Incremental Learningcited by 251
  9. 2021Few-Shot Incremental Learning with Continually Evolved Classifierscited by 187
  10. 2020Negative Margin Matters: Understanding Margin in Few-shot Classificationcited by 129
  11. 2022Forward Compatible Few-Shot Class-Incremental Learningcited by 127
  12. 2014Return of the Devil in the Details: Delving Deep into Convolutional Netscited by 122
  13. 2021Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minimacited by 118
  14. 2023Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learningcited by 116
  15. 2021Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learningcited by 115
  16. 2021Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learningcited by 96
  17. 2022Constrained Few-shot Class-incremental Learningcited by 94
  18. 2022Few-Shot Class-Incremental Learning by Sampling Multi-Phase Taskscited by 88
  19. 2022Few-Shot Class-Incremental Learning from an Open-Set Perspectivecited by 84
  20. 2021A Broad Study on the Transferability of Visual Representations with Contrastive Learningcited by 78
  21. 2020Why Do Better Loss Functions Lead to Less Transferable Features?cited by 73
  22. 2022Few-Shot Class-Incremental Learning via Entropy-Regularized Data-Free Replaycited by 72
  23. 2021Subspace Regularizers for Few-Shot Class Incremental Learningcited by 56
  24. 2023Learning with Fantasy: Semantic-Aware Virtual Contrastive Constraint for Few-Shot Class-Incremental Learningcited by 55
  25. 2022Margin-Based Few-Shot Class-Incremental Learning with Class-Level Overfitting Mitigationcited by 45
  26. 2022Perfectly Balanced: Improving Transfer and Robustness of Supervised Contrastive Learningcited by 40
  27. 2018Ring loss: Convex Feature Normalization for Face Recognitioncited by 39
  28. 2023Which Features are Learnt by Contrastive Learning? On the Role of Simplicity Bias in Class Collapse and Feature Suppressioncited by 30
  29. 2021GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learningcited by 26
  30. 2023S3C: Self-Supervised Stochastic Classifiers for Few-Shot Class-Incremental Learningcited by 23
  31. 2022Discriminability-Transferability Trade-Off: An Information-Theoretic Perspectivecited by 13
  32. 2022Information bottleneck theory of high-dimensional regression: relevancy, efficiency and optimalitycited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 1.

11 papers.

  1. 2025Tripartite Weight-Space Ensemble for Few-Shot Class-Incremental Learningcited by 2
  2. 2025Continuous Knowledge-Preserving Decomposition with Adaptive Layer Selection for Few-Shot Class-Incremental Learningcited by 1
  3. 2025Beyond Synthetic Replays: Turning Diffusion Features into Few-Shot Class-Incremental Learning Knowledgecited by 1
  4. 2025Consistency-Driven Calibration and Matching for Few-Shot Class-Incremental Learningcited by 1
  5. 2025Adaptive Additive Parameter Updates of Vision Transformers for Few-Shot Continual Learningcited by 0
  6. 2025MoTiC: Momentum Tightness and Contrast for Few-Shot Class-Incremental Learningcited by 0
  7. 2025Diffusion-Classifier Synergy: Reward-Aligned Learning via Mutual Boosting Loop for FSCILcited by 0
  8. 2025SAFA-SNN: Sparsity-Aware On-Device Few-Shot Class-Incremental Learning with Fast-Adaptive Structure of Spiking Neural Networkcited by 0
  9. 2025Breaking Forgetting: Training-Free Few-Shot Class-Incremental Learning via Conditional Diffusioncited by 0
  10. 2025Automatic Attack Discovery for Few-Shot Class-Incremental Learning via Large Language Modelscited by 0
  11. 2026Purification and Regulation: Comorbidity-Aware Multi-Label Few-Shot Learning for Medical Image Classificationcited by 0