# Incremental Few-Shot Semantic Segmentation via Embedding Adaptive-Update and Hyper-class Representation

arXiv: https://arxiv.org/abs/2207.12964 · HTML: https://mlsanity.com/paper/2207.12964/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 49 references that are in the map.
- First posted on arXiv 26 Jul 2022.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Meta-Learning with Differentiable Convex Optimization](https://mlsanity.com/paper/1904.03758/) (2019)
- [One-Shot Learning for Semantic Segmentation](https://mlsanity.com/paper/1709.03410/) (2017)
- [Context Encoding for Semantic Segmentation](https://mlsanity.com/paper/1803.08904/) (2018)
- [Learning without Memorizing](https://mlsanity.com/paper/1811.08051/) (2018)
- [CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learning](https://mlsanity.com/paper/1903.02351/) (2019)
- [Adaptive Prototype Learning and Allocation for Few-Shot Segmentation](https://mlsanity.com/paper/2104.01893/) (2021)
- [Modeling the Background for Incremental Learning in Semantic Segmentation](https://mlsanity.com/paper/2002.00718/) (2020)
- [Few-Shot Incremental Learning with Continually Evolved Classifiers](https://mlsanity.com/paper/2104.03047/) (2021)
- [Simultaneous Detection and Segmentation](https://mlsanity.com/paper/1407.1808/) (2014)
- [A continual learning survey: Defying forgetting in classification tasks](https://mlsanity.com/paper/1909.08383/) (2019)
- [Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2103.04059/) (2021)
- [CRNet: Cross-Reference Networks for Few-Shot Segmentation](https://mlsanity.com/paper/2003.10658/) (2020)
- [Learning Dynamic Routing for Semantic Segmentation](https://mlsanity.com/paper/2003.10401/) (2020)
- [Incremental Few-Shot Learning with Attention Attractor Networks](https://mlsanity.com/paper/1810.07218/) (2018)
- [Incremental Few-Shot Instance Segmentation](https://mlsanity.com/paper/2105.05312/) (2021)
- [Knowledge Distillation for Incremental Learning in Semantic Segmentation](https://mlsanity.com/paper/1911.03462/) (2019)
- [XtarNet: Learning to Extract Task-Adaptive Representation for Incremental Few-Shot Learning](https://mlsanity.com/paper/2003.08561/) (2020)

## Built on by (most cited first, 16 of 16)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [A Survey on Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2304.08130/) (2023)
- [M3Net: Multi-view Encoding, Matching, and Fusion for Few-shot Fine-grained Action Recognition](https://mlsanity.com/paper/2308.03063/) (2023)
- [A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Application](https://mlsanity.com/paper/2310.14277/) (2023)
- [kNN-CLIP: Retrieval Enables Training-Free Segmentation on Continually Expanding Large Vocabularies](https://mlsanity.com/paper/2404.09447/) (2024)
- [Advancing Incremental Few-shot Semantic Segmentation via Semantic-guided Relation Alignment and Adaptation](https://mlsanity.com/paper/2305.10868/) (2023)
- [PrecTime: A Deep Learning Architecture for Precise Time Series Segmentation in Industrial Manufacturing Operations](https://mlsanity.com/paper/2302.10182/) (2023)
- [Organizing Background to Explore Latent Classes for Incremental Few-shot Semantic Segmentation](https://mlsanity.com/paper/2405.19568/) (2024)
- [Multi-Content Interaction Network for Few-Shot Segmentation](https://mlsanity.com/paper/2303.06304/) (2023)
- [Few-shot Class-Incremental Semantic Segmentation via Pseudo-Labeling and Knowledge Distillation](https://mlsanity.com/paper/2308.02790/) (2023)
- [Decoupling Continual Semantic Segmentation](https://mlsanity.com/paper/2508.05065/) (2025)
- [Predictive Clustering of Vessel Behavior Based on Hierarchical Trajectory Representation](https://mlsanity.com/paper/2403.08838/) (2024)
- [Ranking-aware Continual Learning for LiDAR Place Recognition](https://mlsanity.com/paper/2505.07198/) (2025)
- [Task Consistent Prototype Learning for Incremental Few-shot Semantic Segmentation](https://mlsanity.com/paper/2410.13094/) (2024)
- [Evolving Without Ending: Unifying Multimodal Incremental Learning for Continual Panoptic Perception](https://mlsanity.com/paper/2601.15643/) (2026)

## Abstract (arXiv)

Incremental few-shot semantic segmentation (IFSS) targets at incrementally expanding model's capacity to segment new class of images supervised by only a few samples. However, features learned on old classes could significantly drift, causing catastrophic forgetting. Moreover, few samples for pixel-level segmentation on new classes lead to notorious overfitting issues in each learning session. In this paper, we explicitly represent class-based knowledge for semantic segmentation as a category embedding and a hyper-class embedding, where the former describes exclusive semantical properties, and the latter expresses hyper-class knowledge as class-shared semantic properties. Aiming to solve IFSS problems, we present EHNet, i.e., Embedding adaptive-update and Hyper-class representation Network from two aspects. First, we propose an embedding adaptive-update strategy to avoid feature drift, which maintains old knowledge by hyper-class representation, and adaptively update category embeddings with a class-attention scheme to involve new classes learned in individual sessions. Second, to resist overfitting issues caused by few training samples, a hyper-class embedding is learned by clustering all category embeddings for initialization and aligned with category embedding of the new class for enhancement, where learned knowledge assists to learn new knowledge, thus alleviating performance dependence on training data scale. Significantly, these two designs provide representation capability for classes with sufficient semantics and limited biases, enabling to perform segmentation tasks requiring high semantic dependence. Experiments on PASCAL-5i and COCO datasets show that EHNet achieves new state-of-the-art performance with remarkable advantages.