# Learning with Fantasy: Semantic-Aware Virtual Contrastive Constraint for Few-Shot Class-Incremental Learning

arXiv: https://arxiv.org/abs/2304.00426 · HTML: https://mlsanity.com/paper/2304.00426/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 59 references that are in the map.
- First posted on arXiv 2 Apr 2023.

## Builds on (most cited first, 30 of 45)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Learning to Prompt for Continual Learning](https://mlsanity.com/paper/2112.08654/) (2021)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Class-incremental learning: survey and performance evaluation on image classification](https://mlsanity.com/paper/2010.15277/) (2020)
- [TADAM: Task dependent adaptive metric for improved few-shot learning](https://mlsanity.com/paper/1805.10123/) (2018)
- [Meta-SGD: Learning to Learn Quickly for Few-Shot Learning](https://mlsanity.com/paper/1707.09835/) (2017)
- [DER: Dynamically Expandable Representation for Class Incremental Learning](https://mlsanity.com/paper/2103.16788/) (2021)
- [Large-Margin Softmax Loss for Convolutional Neural Networks](https://mlsanity.com/paper/1612.02295/) (2016)
- [Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2004.10956/) (2020)
- [Few-Shot Incremental Learning with Continually Evolved Classifiers](https://mlsanity.com/paper/2104.03047/) (2021)
- [Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting](https://mlsanity.com/paper/1802.02950/) (2018)
- [Forward Compatible Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2203.06953/) (2022)
- [Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima](https://mlsanity.com/paper/2111.01549/) (2021)

## Built on by (most cited first, 30 of 55)

- [A Survey on Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2304.08130/) (2023)
- [Few-shot Class-incremental Learning for Classification and Object Detection: A Survey](https://mlsanity.com/paper/2308.06764/) (2023)
- [Compositional Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2405.17022/) (2024)
- [Few-Shot Class Incremental Learning with Attention-Aware Self-Adaptive Prompt](https://mlsanity.com/paper/2403.09857/) (2024)
- [Rethinking Few-shot Class-incremental Learning: Learning from Yourself](https://mlsanity.com/paper/2407.07468/) (2024)
- [Learning Prompt with Distribution-Based Feature Replay for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2401.01598/) (2024)
- [Human-inspired Perspectives: A Survey on AI Long-term Memory](https://mlsanity.com/paper/2411.00489/) (2024)
- [CLOSER: Towards Better Representation Learning for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2410.05627/) (2024)
- [Continual Adapter Tuning with Semantic Shift Compensation for Class-Incremental Learning](https://mlsanity.com/paper/2403.19979/) (2024)
- [Calibrating Higher-Order Statistics for Few-Shot Class-Incremental Learning with Pre-trained Vision Transformers](https://mlsanity.com/paper/2404.06622/) (2024)
- [Revisiting Pool-based Prompt Learning for Few-shot Class-incremental Learning](https://mlsanity.com/paper/2507.09183/) (2025)
- [Contrastive Augmented Graph2Graph Memory Interaction for Few Shot Continual Learning](https://mlsanity.com/paper/2403.04140/) (2024)
- [Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2407.06136/) (2024)
- [Towards Few-Shot Learning in the Open World: A Review and Beyond](https://mlsanity.com/paper/2408.09722/) (2024)
- [Image-Object-Specific Prompt Learning for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2309.02833/) (2023)
- [A streamlined Approach to Multimodal Few-Shot Class Incremental Learning for Fine-Grained Datasets](https://mlsanity.com/paper/2403.06295/) (2024)
- [Latest Advancements Towards Catastrophic Forgetting under Data Scarcity: A Comprehensive Survey on Few-Shot Class Incremental Learning](https://mlsanity.com/paper/2502.08181/) (2025)
- [12 mJ per Class On-Device Online Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2403.07851/) (2024)
- [Inductive Graph Few-shot Class Incremental Learning](https://mlsanity.com/paper/2411.06634/) (2024)
- [Learn by Reasoning: Analogical Weight Generation for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2503.21258/) (2025)
- [A Bag of Tricks for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2403.14392/) (2024)
- [I2CANSAY:Inter-Class Analogical Augmentation and Intra-Class Significance Analysis for Non-Exemplar Online Task-Free Continual Learning](https://mlsanity.com/paper/2404.13576/) (2024)
- [Covariance-based Space Regularization for Few-shot Class Incremental Learning](https://mlsanity.com/paper/2411.01172/) (2024)
- [MultiCo3D: Multi-Label Voxel Contrast for One-Shot Incremental Segmentation of 3D Neuroimages](https://mlsanity.com/paper/2503.06598/) (2025)
- [Tripartite Weight-Space Ensemble for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2506.15720/) (2025)
- [Can Synthetic Images Conquer Forgetting? Beyond Unexplored Doubts in Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2507.13739/) (2025)
- [Bias Mitigating Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2402.00481/) (2024)
- [Controllable Relation Disentanglement for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2403.11070/) (2024)
- [Consistency-Guided Asynchronous Contrastive Tuning for Few-Shot Class-Incremental Tuning of Foundation Models](https://mlsanity.com/paper/2405.16625/) (2024)
- [Distribution-Level Memory Recall for Continual Learning: Preserving Knowledge and Avoiding Confusion](https://mlsanity.com/paper/2408.02695/) (2024)

## Abstract (arXiv)

Few-shot class-incremental learning (FSCIL) aims at learning to classify new classes continually from limited samples without forgetting the old classes. The mainstream framework tackling FSCIL is first to adopt the cross-entropy (CE) loss for training at the base session, then freeze the feature extractor to adapt to new classes. However, in this work, we find that the CE loss is not ideal for the base session training as it suffers poor class separation in terms of representations, which further degrades generalization to novel classes. One tempting method to mitigate this problem is to apply an additional naive supervised contrastive learning (SCL) in the base session. Unfortunately, we find that although SCL can create a slightly better representation separation among different base classes, it still struggles to separate base classes and new classes. Inspired by the observations made, we propose Semantic-Aware Virtual Contrastive model (SAVC), a novel method that facilitates separation between new classes and base classes by introducing virtual classes to SCL. These virtual classes, which are generated via pre-defined transformations, not only act as placeholders for unseen classes in the representation space, but also provide diverse semantic information. By learning to recognize and contrast in the fantasy space fostered by virtual classes, our SAVC significantly boosts base class separation and novel class generalization, achieving new state-of-the-art performance on the three widely-used FSCIL benchmark datasets. Code is available at: https://github.com/zysong0113/SAVC.