# S3C: Self-Supervised Stochastic Classifiers for Few-Shot Class-Incremental Learning

arXiv: https://arxiv.org/abs/2307.02246 · HTML: https://mlsanity.com/paper/2307.02246/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 51 references that are in the map.
- First posted on arXiv 5 Jul 2023.

## Builds on (most cited first, 27 of 27)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [Learning Deconvolution Network for Semantic Segmentation](https://mlsanity.com/paper/1505.04366/) (2015)
- [S4L: Self-Supervised Semi-Supervised Learning](https://mlsanity.com/paper/1905.03670/) (2019)
- [Maintaining Discrimination and Fairness in Class Incremental Learning](https://mlsanity.com/paper/1911.07053/) (2019)
- [Zero-Shot Learning -- The Good, the Bad and the Ugly](https://mlsanity.com/paper/1703.04394/) (2017)
- [Learning Representations for Automatic Colorization](https://mlsanity.com/paper/1603.06668/) (2016)
- [Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2004.10956/) (2020)
- [Semantic Drift Compensation for Class-Incremental Learning](https://mlsanity.com/paper/2004.00440/) (2020)
- [Rethinking the Value of Labels for Improving Class-Imbalanced Learning](https://mlsanity.com/paper/2006.07529/) (2020)
- [Few-Shot Incremental Learning with Continually Evolved Classifiers](https://mlsanity.com/paper/2104.03047/) (2021)
- [Boosting Few-Shot Visual Learning with Self-Supervision](https://mlsanity.com/paper/1906.05186/) (2019)
- [Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima](https://mlsanity.com/paper/2111.01549/) (2021)
- [Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2103.04059/) (2021)
- [Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2107.08918/) (2021)
- [Few-Shot Lifelong Learning](https://mlsanity.com/paper/2103.00991/) (2021)
- [Subspace Regularizers for Few-Shot Class Incremental Learning](https://mlsanity.com/paper/2110.07059/) (2021)
- [ScaIL: Classifier Weights Scaling for Class Incremental Learning](https://mlsanity.com/paper/2001.05755/) (2020)
- [Graph Few-shot Class-incremental Learning](https://mlsanity.com/paper/2112.12819/) (2021)
- [Few-Shot and Continual Learning with Attentive Independent Mechanisms](https://mlsanity.com/paper/2107.14053/) (2021)

## Built on by (most cited first, 23 of 23)

- [Few-shot Class-incremental Learning for Classification and Object Detection: A Survey](https://mlsanity.com/paper/2308.06764/) (2023)
- [Rethinking Few-shot Class-incremental Learning: Learning from Yourself](https://mlsanity.com/paper/2407.07468/) (2024)
- [Continual Unlearning for Foundational Text-to-Image Models without Generalization Erosion](https://mlsanity.com/paper/2503.13769/) (2025)
- [CLOSER: Towards Better Representation Learning for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2410.05627/) (2024)
- [Calibrating Higher-Order Statistics for Few-Shot Class-Incremental Learning with Pre-trained Vision Transformers](https://mlsanity.com/paper/2404.06622/) (2024)
- [Towards Few-Shot Learning in the Open World: A Review and Beyond](https://mlsanity.com/paper/2408.09722/) (2024)
- [Few-Shot Class Incremental Learning via Robust Transformer Approach](https://mlsanity.com/paper/2405.05984/) (2024)
- [I2CANSAY:Inter-Class Analogical Augmentation and Intra-Class Significance Analysis for Non-Exemplar Online Task-Free Continual Learning](https://mlsanity.com/paper/2404.13576/) (2024)
- [Tripartite Weight-Space Ensemble for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2506.15720/) (2025)
- [Continual Knowledge Consolidation LORA for Domain Incremental Learning](https://mlsanity.com/paper/2510.16077/) (2025)
- [Bias Mitigating Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2402.00481/) (2024)
- [Unsupervised Few-Shot Continual Learning for Remote Sensing Image Scene Classification](https://mlsanity.com/paper/2406.18574/) (2024)
- [On Distilling the Displacement Knowledge for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2412.11017/) (2024)
- [Does Prior Data Matter? Exploring Joint Training in the Context of Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2503.10003/) (2025)
- [Big-model Driven Few-shot Continual Learning](https://mlsanity.com/paper/2309.00862/) (2023)
- [AggSS: An Aggregated Self-Supervised Approach for Class-Incremental Learning](https://mlsanity.com/paper/2408.04347/) (2024)
- [Few-Shot Class-Incremental Learning with Non-IID Decentralized Data](https://mlsanity.com/paper/2409.11657/) (2024)
- [Embedding Space Allocation with Angle-Norm Joint Classifiers for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2411.09250/) (2024)
- [MEGA: Second-Order Gradient Alignment for Catastrophic Forgetting Mitigation in GFSCIL](https://mlsanity.com/paper/2504.13691/) (2025)
- [Few-Shot Class-Incremental Learning For Efficient SAR Automatic Target Recognition](https://mlsanity.com/paper/2505.19565/) (2025)
- [Multiple Stochastic Prompt Tuning for Few-shot Adaptation under Extreme Domain Shift](https://mlsanity.com/paper/2506.03926/) (2025)
- [SAFA-SNN: Sparsity-Aware On-Device Few-Shot Class-Incremental Learning with Fast-Adaptive Structure of Spiking Neural Network](https://mlsanity.com/paper/2510.03648/) (2025)
- [Breaking Forgetting: Training-Free Few-Shot Class-Incremental Learning via Conditional Diffusion](https://mlsanity.com/paper/2511.18516/) (2025)

## Abstract (arXiv)

Few-shot class-incremental learning (FSCIL) aims to learn progressively about new classes with very few labeled samples, without forgetting the knowledge of already learnt classes. FSCIL suffers from two major challenges: (i) over-fitting on the new classes due to limited amount of data, (ii) catastrophically forgetting about the old classes due to unavailability of data from these classes in the incremental stages. In this work, we propose a self-supervised stochastic classifier (S3C) to counter both these challenges in FSCIL. The stochasticity of the classifier weights (or class prototypes) not only mitigates the adverse effect of absence of large number of samples of the new classes, but also the absence of samples from previously learnt classes during the incremental steps. This is complemented by the self-supervision component, which helps to learn features from the base classes which generalize well to unseen classes that are encountered in future, thus reducing catastrophic forgetting. Extensive evaluation on three benchmark datasets using multiple evaluation metrics show the effectiveness of the proposed framework. We also experiment on two additional realistic scenarios of FSCIL, namely where the number of annotated data available for each of the new classes can be different, and also where the number of base classes is much lesser, and show that the proposed S3C performs significantly better than the state-of-the-art for all these challenging scenarios.