# Incremental Learning in Semantic Segmentation from Image Labels

arXiv: https://arxiv.org/abs/2112.01882 · HTML: https://mlsanity.com/paper/2112.01882/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 59 of its 70 references that are in the map.
- First posted on arXiv 3 Dec 2021.

## Builds on (most cited first, 30 of 59)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1604.05144/) (2016)
- [Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/1803.10464/) (2018)
- [Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2004.04581/) (2020)
- [What's the Point: Semantic Segmentation with Point Supervision](https://mlsanity.com/paper/1506.02106/) (2015)
- [Learning without Memorizing](https://mlsanity.com/paper/1811.08051/) (2018)
- [Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations](https://mlsanity.com/paper/1904.05044/) (2019)
- [BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1503.01640/) (2015)
- [Simple Does It: Weakly Supervised Instance and Semantic Segmentation](https://mlsanity.com/paper/1603.07485/) (2016)
- [Mnemonics Training: Multi-Class Incremental Learning without Forgetting](https://mlsanity.com/paper/2002.10211/) (2020)
- [Does label smoothing mitigate label noise?](https://mlsanity.com/paper/2003.02819/) (2020)
- [Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach](https://mlsanity.com/paper/1703.08448/) (2017)
- [Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation](https://mlsanity.com/paper/1603.06098/) (2016)
- [The Computational Limits of Deep Learning](https://mlsanity.com/paper/2007.05558/) (2020)
- [Modeling the Background for Incremental Learning in Semantic Segmentation](https://mlsanity.com/paper/2002.00718/) (2020)

## Built on by (most cited first, 30 of 39)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need](https://mlsanity.com/paper/2303.07338/) (2023)
- [Unmasking Anomalies in Road-Scene Segmentation](https://mlsanity.com/paper/2307.13316/) (2023)
- [Audio-Visual Class-Incremental Learning](https://mlsanity.com/paper/2308.11073/) (2023)
- [Semantic Image Segmentation: Two Decades of Research](https://mlsanity.com/paper/2302.06378/) (2023)
- [CoMFormer: Continual Learning in Semantic and Panoptic Segmentation](https://mlsanity.com/paper/2211.13999/) (2022)
- [Modeling Missing Annotations for Incremental Learning in Object Detection](https://mlsanity.com/paper/2204.08766/) (2022)
- [A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Application](https://mlsanity.com/paper/2310.14277/) (2023)
- [Mask2Anomaly: Mask Transformer for Universal Open-set Segmentation](https://mlsanity.com/paper/2309.04573/) (2023)
- [Foundation Model Drives Weakly Incremental Learning for Semantic Segmentation](https://mlsanity.com/paper/2302.14250/) (2023)
- [Learning with Style: Continual Semantic Segmentation Across Tasks and Domains](https://mlsanity.com/paper/2210.07016/) (2022)
- [Continual Learning for LiDAR Semantic Segmentation: Class-Incremental and Coarse-to-Fine strategies on Sparse Data](https://mlsanity.com/paper/2304.03980/) (2023)
- [Continual Segmentation with Disentangled Objectness Learning and Class Recognition](https://mlsanity.com/paper/2403.03477/) (2024)
- [Online Unsupervised Domain Adaptation for Person Re-identification](https://mlsanity.com/paper/2205.04383/) (2022)
- [RaSP: Relation-aware Semantic Prior for Weakly Supervised Incremental Segmentation](https://mlsanity.com/paper/2305.19879/) (2023)
- [Continual Road-Scene Semantic Segmentation via Feature-Aligned Symmetric Multi-Modal Network](https://mlsanity.com/paper/2308.04702/) (2023)
- [CoMBO: Conflict Mitigation via Branched Optimization for Class Incremental Segmentation](https://mlsanity.com/paper/2504.04156/) (2025)
- [Learning from the Web: Language Drives Weakly-Supervised Incremental Learning for Semantic Segmentation](https://mlsanity.com/paper/2407.13363/) (2024)
- [Causes of Catastrophic Forgetting in Class-Incremental Semantic Segmentation](https://mlsanity.com/paper/2209.08010/) (2022)
- [Principles of Forgetting in Domain-Incremental Semantic Segmentation in Adverse Weather Conditions](https://mlsanity.com/paper/2303.14115/) (2023)
- [RECALL+: Adversarial Web-based Replay for Continual Learning in Semantic Segmentation](https://mlsanity.com/paper/2309.10479/) (2023)
- [Scribbles for All: Benchmarking Scribble Supervised Segmentation Across Datasets](https://mlsanity.com/paper/2408.12489/) (2024)
- [Effects of Architectures on Continual Semantic Segmentation](https://mlsanity.com/paper/2302.10718/) (2023)
- [Source-Guided Similarity Preservation for Online Person Re-Identification](https://mlsanity.com/paper/2402.15206/) (2024)
- [Tendency-driven Mutual Exclusivity for Weakly Supervised Incremental Semantic Segmentation](https://mlsanity.com/paper/2404.11981/) (2024)
- [Towards Label-Efficient Incremental Learning: A Survey](https://mlsanity.com/paper/2302.00353/) (2023)
- [PANDAS: Prototype-based Novel Class Discovery and Detection](https://mlsanity.com/paper/2402.17420/) (2024)
- [Pixel-Level Domain Adaptation: A New Perspective for Enhancing Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2408.02039/) (2024)
- [Continual Multiple Instance Learning with Enhanced Localization for Histopathological Whole Slide Image Analysis](https://mlsanity.com/paper/2507.02395/) (2025)

## Abstract (arXiv)

Although existing semantic segmentation approaches achieve impressive results, they still struggle to update their models incrementally as new categories are uncovered. Furthermore, pixel-by-pixel annotations are expensive and time-consuming. This paper proposes a novel framework for Weakly Incremental Learning for Semantic Segmentation, that aims at learning to segment new classes from cheap and largely available image-level labels. As opposed to existing approaches, that need to generate pseudo-labels offline, we use an auxiliary classifier, trained with image-level labels and regularized by the segmentation model, to obtain pseudo-supervision online and update the model incrementally. We cope with the inherent noise in the process by using soft-labels generated by the auxiliary classifier. We demonstrate the effectiveness of our approach on the Pascal VOC and COCO datasets, outperforming offline weakly-supervised methods and obtaining results comparable with incremental learning methods with full supervision. Code can be found at https://github.com/fcdl94/WILSON.