# Fairness Continual Learning Approach to Semantic Scene Understanding in Open-World Environments

arXiv: https://arxiv.org/abs/2305.15700 · HTML: https://mlsanity.com/paper/2305.15700/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 52 references that are in the map.
- First posted on arXiv 25 May 2023.

## Builds on (most cited first, 30 of 39)

- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1811.12833/) (2018)
- [Balanced Meta-Softmax for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2007.10740/) (2020)
- [DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation](https://mlsanity.com/paper/2111.14887/) (2021)
- [Towards Open World Object Detection](https://mlsanity.com/paper/2103.02603/) (2021)
- [Maintaining Discrimination and Fairness in Class Incremental Learning](https://mlsanity.com/paper/1911.07053/) (2019)
- [Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2101.10979/) (2021)
- [Incremental Learning of Object Detectors without Catastrophic Forgetting](https://mlsanity.com/paper/1708.06977/) (2017)
- [Modeling the Background for Incremental Learning in Semantic Segmentation](https://mlsanity.com/paper/2002.00718/) (2020)
- [PLOP: Learning without Forgetting for Continual Semantic Segmentation](https://mlsanity.com/paper/2011.11390/) (2020)
- [Self-supervised Augmentation Consistency for Adapting Semantic Segmentation](https://mlsanity.com/paper/2105.00097/) (2021)
- [Seesaw Loss for Long-Tailed Instance Segmentation](https://mlsanity.com/paper/2008.10032/) (2020)
- [Incremental Learning Techniques for Semantic Segmentation](https://mlsanity.com/paper/1907.13372/) (2019)
- [DADA: Depth-aware Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1904.01886/) (2019)
- [Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations](https://mlsanity.com/paper/2103.06342/) (2021)
- [Representation Compensation Networks for Continual Semantic Segmentation](https://mlsanity.com/paper/2203.05402/) (2022)
- [SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning](https://mlsanity.com/paper/2106.11562/) (2021)
- [Continual Adaptation of Visual Representations via Domain Randomization and Meta-learning](https://mlsanity.com/paper/2012.04324/) (2020)
- [Deep Metric Learning for Open World Semantic Segmentation](https://mlsanity.com/paper/2108.04562/) (2021)
- [DropLoss for Long-Tail Instance Segmentation](https://mlsanity.com/paper/2104.06402/) (2021)
- [BiMaL: Bijective Maximum Likelihood Approach to Domain Adaptation in Semantic Scene Segmentation](https://mlsanity.com/paper/2108.03267/) (2021)

## Built on by (most cited first, 16 of 16)

- [Insect-Foundation: A Foundation Model and Large-scale 1M Dataset for Visual Insect Understanding](https://mlsanity.com/paper/2311.15206/) (2023)
- [A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Application](https://mlsanity.com/paper/2310.14277/) (2023)
- [FALCON: Fairness Learning via Contrastive Attention Approach to Continual Semantic Scene Understanding](https://mlsanity.com/paper/2311.15965/) (2023)
- [Continual Learning for Smart City: A Survey](https://mlsanity.com/paper/2404.00983/) (2024)
- [Quantum-Brain: Quantum-Inspired Neural Network Approach to Vision-Brain Understanding](https://mlsanity.com/paper/2411.13378/) (2024)
- [Insect-Foundation: A Foundation Model and Large Multimodal Dataset for Vision-Language Insect Understanding](https://mlsanity.com/paper/2502.09906/) (2025)
- [Towards Incremental Learning in Large Language Models: A Critical Review](https://mlsanity.com/paper/2404.18311/) (2024)
- [EAGLE: Efficient Adaptive Geometry-based Learning in Cross-view Understanding](https://mlsanity.com/paper/2406.01429/) (2024)
- [Enhancing Group Fairness in Online Settings Using Oblique Decision Forests](https://mlsanity.com/paper/2310.11401/) (2023)
- [MANGO: Multimodal Attention-based Normalizing Flow Approach to Fusion Learning](https://mlsanity.com/paper/2508.10133/) (2025)
- [Exploring Open-world Continual Learning with Knowns-Unknowns Knowledge Transfer](https://mlsanity.com/paper/2502.20124/) (2025)
- [Decoupling Continual Semantic Segmentation](https://mlsanity.com/paper/2508.05065/) (2025)
- [CLIFF: Continual Learning for Incremental Flake Features in 2D Material Identification](https://mlsanity.com/paper/2508.17261/) (2025)
- [φ-DPO: Fairness Direct Preference Optimization Approach to Continual Learning in Large Multimodal Models](https://mlsanity.com/paper/2602.22601/) (2026)
- [OpenQlaw: An Agentic AI Assistant for Analysis of 2D Quantum Materials](https://mlsanity.com/paper/2603.17043/) (2026)
- [Fair Class-Incremental Learning using Sample Weighting](https://mlsanity.com/paper/2410.01324/) (2024)

## Abstract (arXiv)

Continual semantic segmentation aims to learn new classes while maintaining the information from the previous classes. Although prior studies have shown impressive progress in recent years, the fairness concern in the continual semantic segmentation needs to be better addressed. Meanwhile, fairness is one of the most vital factors in deploying the deep learning model, especially in human-related or safety applications. In this paper, we present a novel Fairness Continual Learning approach to the semantic segmentation problem. In particular, under the fairness objective, a new fairness continual learning framework is proposed based on class distributions. Then, a novel Prototypical Contrastive Clustering loss is proposed to address the significant challenges in continual learning, i.e., catastrophic forgetting and background shift. Our proposed loss has also been proven as a novel, generalized learning paradigm of knowledge distillation commonly used in continual learning. Moreover, the proposed Conditional Structural Consistency loss further regularized the structural constraint of the predicted segmentation. Our proposed approach has achieved State-of-the-Art performance on three standard scene understanding benchmarks, i.e., ADE20K, Cityscapes, and Pascal VOC, and promoted the fairness of the segmentation model.