# Uncertainty-Aware Distillation for Semi-Supervised Few-Shot Class-Incremental Learning

arXiv: https://arxiv.org/abs/2301.09964 · HTML: https://mlsanity.com/paper/2301.09964/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 66 references that are in the map.
- First posted on arXiv 24 Jan 2023.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling](https://mlsanity.com/paper/2110.08263/) (2021)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning](https://mlsanity.com/paper/2101.06329/) (2021)
- [Label Propagation for Deep Semi-supervised Learning](https://mlsanity.com/paper/1904.04717/) (2019)
- [Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2004.10956/) (2020)
- [Few-Shot Incremental Learning with Continually Evolved Classifiers](https://mlsanity.com/paper/2104.03047/) (2021)
- [Task-Agnostic Meta-Learning for Few-shot Learning](https://mlsanity.com/paper/1805.07722/) (2018)
- [Forward Compatible Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2203.06953/) (2022)
- [Learning What Not to Segment: A New Perspective on Few-Shot Segmentation](https://mlsanity.com/paper/2203.07615/) (2022)
- [Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2103.04059/) (2021)
- [Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2107.08918/) (2021)
- [Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriors](https://mlsanity.com/paper/1603.04733/) (2016)
- [Graph-based Semi-supervised Learning: A Comprehensive Review](https://mlsanity.com/paper/2102.13303/) (2021)

## Built on by (most cited first, 16 of 16)

- [Few-shot Class-incremental Learning for Classification and Object Detection: A Survey](https://mlsanity.com/paper/2308.06764/) (2023)
- [Rethinking Few-shot Class-incremental Learning: Learning from Yourself](https://mlsanity.com/paper/2407.07468/) (2024)
- [Rehearsal-Free Domain Continual Face Anti-Spoofing: Generalize More and Forget Less](https://mlsanity.com/paper/2303.09914/) (2023)
- [Revisiting Pool-based Prompt Learning for Few-shot Class-incremental Learning](https://mlsanity.com/paper/2507.09183/) (2025)
- [Uncertainty-informed Mutual Learning for Joint Medical Image Classification and Segmentation](https://mlsanity.com/paper/2303.10049/) (2023)
- ["X of Information'' Continuum: A Survey on AI-Driven Multi-dimensional Metrics for Next-Generation Networked Systems](https://mlsanity.com/paper/2507.19657/) (2025)
- [Towards Few-Shot Learning in the Open World: A Review and Beyond](https://mlsanity.com/paper/2408.09722/) (2024)
- [Branch-Tuning: Balancing Stability and Plasticity for Continual Self-Supervised Learning](https://mlsanity.com/paper/2403.18266/) (2024)
- [Few-Shot Class-Incremental Learning with Prior Knowledge](https://mlsanity.com/paper/2402.01201/) (2024)
- [Divide-and-Conquer for Enhancing Unlabeled Learning, Stability, and Plasticity in Semi-supervised Continual Learning](https://mlsanity.com/paper/2508.05316/) (2025)
- [Controllable Relation Disentanglement for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2403.11070/) (2024)
- [On Distilling the Displacement Knowledge for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2412.11017/) (2024)
- [Knowledge Transfer-Driven Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2306.10942/) (2023)
- [Contextual Representation Anchor Network to Alleviate Selection Bias in Few-Shot Drug Discovery](https://mlsanity.com/paper/2410.20711/) (2024)
- [Ambiguity-Guided Learnable Distribution Calibration for Semi-Supervised Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2507.23237/) (2025)
- [Unsupervised Active Learning via Natural Feature Progressive Framework](https://mlsanity.com/paper/2510.04939/) (2025)

## Abstract (arXiv)

Given a model well-trained with a large-scale base dataset, Few-Shot Class-Incremental Learning (FSCIL) aims at incrementally learning novel classes from a few labeled samples by avoiding overfitting, without catastrophically forgetting all encountered classes previously. Currently, semi-supervised learning technique that harnesses freely-available unlabeled data to compensate for limited labeled data can boost the performance in numerous vision tasks, which heuristically can be applied to tackle issues in FSCIL, i.e., the Semi-supervised FSCIL (Semi-FSCIL). So far, very limited work focuses on the Semi-FSCIL task, leaving the adaptability issue of semi-supervised learning to the FSCIL task unresolved. In this paper, we focus on this adaptability issue and present a simple yet efficient Semi-FSCIL framework named Uncertainty-aware Distillation with Class-Equilibrium (UaD-CE), encompassing two modules UaD and CE. Specifically, when incorporating unlabeled data into each incremental session, we introduce the CE module that employs a class-balanced self-training to avoid the gradual dominance of easy-to-classified classes on pseudo-label generation. To distill reliable knowledge from the reference model, we further implement the UaD module that combines uncertainty-guided knowledge refinement with adaptive distillation. Comprehensive experiments on three benchmark datasets demonstrate that our method can boost the adaptability of unlabeled data with the semi-supervised learning technique in FSCIL tasks.