# Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation

arXiv: https://arxiv.org/abs/2203.14098 · HTML: https://mlsanity.com/paper/2203.14098/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 65 references that are in the map.
- First posted on arXiv 26 Mar 2022.

## Builds on (most cited first, 30 of 54)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [Contrastive Learning of Medical Visual Representations from Paired Images and Text](https://mlsanity.com/paper/2010.00747/) (2020)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation](https://mlsanity.com/paper/1611.06612/) (2016)
- [Contrastive Adaptation Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1901.00976/) (2019)
- [Learning without Memorizing](https://mlsanity.com/paper/1811.08051/) (2018)
- [Exploring Cross-Image Pixel Contrast for Semantic Segmentation](https://mlsanity.com/paper/2101.11939/) (2021)

## Built on by (most cited first, 30 of 37)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [Attribution-aware Weight Transfer: A Warm-Start Initialization for Class-Incremental Semantic Segmentation](https://mlsanity.com/paper/2210.07207/) (2022)
- [A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Application](https://mlsanity.com/paper/2310.14277/) (2023)
- [ECLIPSE: Efficient Continual Learning in Panoptic Segmentation with Visual Prompt Tuning](https://mlsanity.com/paper/2403.20126/) (2024)
- [CoinSeg: Contrast Inter- and Intra- Class Representations for Incremental Segmentation](https://mlsanity.com/paper/2310.06368/) (2023)
- [Inherit with Distillation and Evolve with Contrast: Exploring Class Incremental Semantic Segmentation Without Exemplar Memory](https://mlsanity.com/paper/2309.15413/) (2023)
- [Gradient-Semantic Compensation for Incremental Semantic Segmentation](https://mlsanity.com/paper/2307.10822/) (2023)
- [Strike a Balance in Continual Panoptic Segmentation](https://mlsanity.com/paper/2407.16354/) (2024)
- [Learning with Style: Continual Semantic Segmentation Across Tasks and Domains](https://mlsanity.com/paper/2210.07016/) (2022)
- [Incremental Learning for Heterogeneous Structure Segmentation in Brain Tumor MRI](https://mlsanity.com/paper/2305.19404/) (2023)
- [Self-paced Weight Consolidation for Continual Learning](https://mlsanity.com/paper/2307.10845/) (2023)
- [Mitigating Background Shift in Class-Incremental Semantic Segmentation](https://mlsanity.com/paper/2407.11859/) (2024)
- [MRN: Multiplexed Routing Network for Incremental Multilingual Text Recognition](https://mlsanity.com/paper/2305.14758/) (2023)
- [Early Preparation Pays Off: New Classifier Pre-tuning for Class Incremental Semantic Segmentation](https://mlsanity.com/paper/2407.14142/) (2024)
- [Learning at a Glance: Towards Interpretable Data-limited Continual Semantic Segmentation via Semantic-Invariance Modelling](https://mlsanity.com/paper/2407.15429/) (2024)
- [Balanced Residual Distillation Learning for 3D Point Cloud Class-Incremental Semantic Segmentation](https://mlsanity.com/paper/2408.01356/) (2024)
- [CLEO: Continual Learning of Evolving Ontologies](https://mlsanity.com/paper/2407.08411/) (2024)
- [Background Adaptation with Residual Modeling for Exemplar-Free Class-Incremental Semantic Segmentation](https://mlsanity.com/paper/2407.09838/) (2024)
- [Learning from the Web: Language Drives Weakly-Supervised Incremental Learning for Semantic Segmentation](https://mlsanity.com/paper/2407.13363/) (2024)
- [Detecting Wildfires on UAVs with Real-time Segmentation Trained by Larger Teacher Models](https://mlsanity.com/paper/2408.10843/) (2024)
- [RECALL+: Adversarial Web-based Replay for Continual Learning in Semantic Segmentation](https://mlsanity.com/paper/2309.10479/) (2023)
- [Towards A Robust Group-level Emotion Recognition via Uncertainty-Aware Learning](https://mlsanity.com/paper/2310.04306/) (2023)
- [KAC: Kolmogorov-Arnold Classifier for Continual Learning](https://mlsanity.com/paper/2503.21076/) (2025)
- [HPCR: Holistic Proxy-based Contrastive Replay for Online Continual Learning](https://mlsanity.com/paper/2309.15038/) (2023)
- [BACS: Background Aware Continual Semantic Segmentation](https://mlsanity.com/paper/2404.13148/) (2024)
- [CFSSeg: Closed-Form Solution for Class-Incremental Semantic Segmentation of 2D Images and 3D Point Clouds](https://mlsanity.com/paper/2412.10834/) (2024)
- [AMRD: Adaptive Multi-Teacher Relational Distillation for Lightweight Speech Emotion Recognition](https://mlsanity.com/paper/2607.25289/) (2026)
- [Learning Disentangled Label Representations for Multi-label Classification](https://mlsanity.com/paper/2212.01461/) (2022)
- [PARMESAN: Parameter-Free Memory Search and Transduction for Dense Prediction Tasks](https://mlsanity.com/paper/2403.11743/) (2024)

## Abstract (arXiv)

A fundamental and challenging problem in deep learning is catastrophic forgetting, i.e. the tendency of neural networks to fail to preserve the knowledge acquired from old tasks when learning new tasks. This problem has been widely investigated in the research community and several Incremental Learning (IL) approaches have been proposed in the past years. While earlier works in computer vision have mostly focused on image classification and object detection, more recently some IL approaches for semantic segmentation have been introduced. These previous works showed that, despite its simplicity, knowledge distillation can be effectively employed to alleviate catastrophic forgetting. In this paper, we follow this research direction and, inspired by recent literature on contrastive learning, we propose a novel distillation framework, Uncertainty-aware Contrastive Distillation (method). In a nutshell, method~is operated by introducing a novel distillation loss that takes into account all the images in a mini-batch, enforcing similarity between features associated to all the pixels from the same classes, and pulling apart those corresponding to pixels from different classes. In order to mitigate catastrophic forgetting, we contrast features of the new model with features extracted by a frozen model learned at the previous incremental step. Our experimental results demonstrate the advantage of the proposed distillation technique, which can be used in synergy with previous IL approaches, and leads to state-of-art performance on three commonly adopted benchmarks for incremental semantic segmentation. The code is available at urlhttps://github.com/ygjwd12345/UCD.