# Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Data

arXiv: https://arxiv.org/abs/1810.12448 · HTML: https://mlsanity.com/paper/1810.12448/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2018.

## Built on by (most cited first, 30 of 35)

- [Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations](https://mlsanity.com/paper/2103.06342/) (2021)
- [Replay in Deep Learning: Current Approaches and Missing Biological Elements](https://mlsanity.com/paper/2104.04132/) (2021)
- [Recent Advances of Continual Learning in Computer Vision: An Overview](https://mlsanity.com/paper/2109.11369/) (2021)
- [Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation](https://mlsanity.com/paper/2203.14098/) (2022)
- [An EM Framework for Online Incremental Learning of Semantic Segmentation](https://mlsanity.com/paper/2108.03613/) (2021)
- [Continual Barlow Twins: continual self-supervised learning for remote sensing semantic segmentation](https://mlsanity.com/paper/2205.11319/) (2022)
- [Attribution-aware Weight Transfer: A Warm-Start Initialization for Class-Incremental Semantic Segmentation](https://mlsanity.com/paper/2210.07207/) (2022)
- [A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Application](https://mlsanity.com/paper/2310.14277/) (2023)
- [Learning Incrementally to Segment Multiple Organs in a CT Image](https://mlsanity.com/paper/2203.02100/) (2022)
- [Continual Learning for Class- and Domain-Incremental Semantic Segmentation](https://mlsanity.com/paper/2209.08023/) (2022)
- [Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation](https://mlsanity.com/paper/2106.15287/) (2021)
- [Continual Attentive Fusion for Incremental Learning in Semantic Segmentation](https://mlsanity.com/paper/2202.00432/) (2022)
- [Towards Unsupervised Open World Semantic Segmentation](https://mlsanity.com/paper/2201.01073/) (2022)
- [Inherit with Distillation and Evolve with Contrast: Exploring Class Incremental Semantic Segmentation Without Exemplar Memory](https://mlsanity.com/paper/2309.15413/) (2023)
- [Incremental Learning for Multi-organ Segmentation with Partially Labeled Datasets](https://mlsanity.com/paper/2103.04526/) (2021)
- [Half-Real Half-Fake Distillation for Class-Incremental Semantic Segmentation](https://mlsanity.com/paper/2104.00875/) (2021)
- [Improving Replay-Based Continual Semantic Segmentation with Smart Data Selection](https://mlsanity.com/paper/2209.09839/) (2022)
- [A Contrastive Distillation Approach for Incremental Semantic Segmentation in Aerial Images](https://mlsanity.com/paper/2112.03814/) (2021)
- [Learning at a Glance: Towards Interpretable Data-limited Continual Semantic Segmentation via Semantic-Invariance Modelling](https://mlsanity.com/paper/2407.15429/) (2024)
- [Detecting and Learning the Unknown in Semantic Segmentation](https://mlsanity.com/paper/2202.08700/) (2022)
- [Causes of Catastrophic Forgetting in Class-Incremental Semantic Segmentation](https://mlsanity.com/paper/2209.08010/) (2022)
- [Targeted Gradient Descent: A Novel Method for Convolutional Neural Networks Fine-tuning and Online-learning](https://mlsanity.com/paper/2109.14729/) (2021)
- [Weakly-supervised continual learning for class-incremental segmentation](https://mlsanity.com/paper/2201.01029/) (2022)
- [Modeling the Background for Incremental and Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2201.13338/) (2022)
- [Decoupling Continual Semantic Segmentation](https://mlsanity.com/paper/2508.05065/) (2025)
- [RSI-Net: Two-Stream Deep Neural Network for Remote Sensing Imagesbased Semantic Segmentation](https://mlsanity.com/paper/2109.09148/) (2021)
- [Reviewing continual learning from the perspective of human-level intelligence](https://mlsanity.com/paper/2111.11964/) (2021)
- [Delving into Transformer for Incremental Semantic Segmentation](https://mlsanity.com/paper/2211.10253/) (2022)
- [Online Clustering of Seafloor Imagery for Interpretation during Long-Term AUV Operations](https://mlsanity.com/paper/2509.06678/) (2025)
- [Overcoming Catastrophic Forgetting by XAI](https://mlsanity.com/paper/2211.14177/) (2022)

## Abstract (arXiv)

In spite of remarkable success of the convolutional neural networks on semantic segmentation, they suffer from catastrophic forgetting: a significant performance drop for the already learned classes when new classes are added on the data, having no annotations for the old classes. We propose an incremental learning methodology, enabling to learn segmenting new classes without hindering dense labeling abilities for the previous classes, although the entire previous data are not accessible. The key points of the proposed approach are adapting the network to learn new as well as old classes on the new training data, and allowing it to remember the previously learned information for the old classes. For adaptation, we keep a frozen copy of the previously trained network, which is used as a memory for the updated network in absence of annotations for the former classes. The updated network minimizes a loss function, which balances the discrepancy between outputs for the previous classes from the memory and updated networks, and the mis-classification rate between outputs for the new classes from the updated network and the new ground-truth. For remembering, we either regularly feed samples from the stored, little fraction of the previous data or use the memory network, depending on whether the new data are collected from completely different geographic areas or from the same city. Our experimental results prove that it is possible to add new classes to the network, while maintaining its performance for the previous classes, despite the whole previous training data are not available.