# FALCON: Fairness Learning via Contrastive Attention Approach to Continual Semantic Scene Understanding

arXiv: https://arxiv.org/abs/2311.15965 · HTML: https://mlsanity.com/paper/2311.15965/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 80 references that are in the map.
- First posted on arXiv 27 Nov 2023.

## Builds on (most cited first, 30 of 53)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling](https://mlsanity.com/paper/2102.06183/) (2021)
- [Balanced Meta-Softmax for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2007.10740/) (2020)
- [DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation](https://mlsanity.com/paper/2111.14887/) (2021)
- [Towards Open World Object Detection](https://mlsanity.com/paper/2103.02603/) (2021)
- [Contrastive Clustering](https://mlsanity.com/paper/2009.09687/) (2020)
- [Parametric Contrastive Learning](https://mlsanity.com/paper/2107.12028/) (2021)
- [Incremental Learning of Object Detectors without Catastrophic Forgetting](https://mlsanity.com/paper/1708.06977/) (2017)
- [Modeling the Background for Incremental Learning in Semantic Segmentation](https://mlsanity.com/paper/2002.00718/) (2020)
- [PLOP: Learning without Forgetting for Continual Semantic Segmentation](https://mlsanity.com/paper/2011.11390/) (2020)
- [Self-supervised Augmentation Consistency for Adapting Semantic Segmentation](https://mlsanity.com/paper/2105.00097/) (2021)
- [Seesaw Loss for Long-Tailed Instance Segmentation](https://mlsanity.com/paper/2008.10032/) (2020)
- [Balanced Contrastive Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2207.09052/) (2022)

## Built on by (most cited first, 13 of 13)

- [Quantum-Brain: Quantum-Inspired Neural Network Approach to Vision-Brain Understanding](https://mlsanity.com/paper/2411.13378/) (2024)
- [EAGLE: Efficient Adaptive Geometry-based Learning in Cross-view Understanding](https://mlsanity.com/paper/2406.01429/) (2024)
- [MANGO: Multimodal Attention-based Normalizing Flow Approach to Fusion Learning](https://mlsanity.com/paper/2508.10133/) (2025)
- [Fairness Evolution in Continual Learning for Medical Imaging](https://mlsanity.com/paper/2406.02480/) (2024)
- [Learning to Evolve: Bayesian-Guided Continual Knowledge Graph Embedding](https://mlsanity.com/paper/2508.02426/) (2025)
- [CLIFF: Continual Learning for Incremental Flake Features in 2D Material Identification](https://mlsanity.com/paper/2508.17261/) (2025)
- [φ-DPO: Fairness Direct Preference Optimization Approach to Continual Learning in Large Multimodal Models](https://mlsanity.com/paper/2602.22601/) (2026)
- [OpenQlaw: An Agentic AI Assistant for Analysis of 2D Quantum Materials](https://mlsanity.com/paper/2603.17043/) (2026)
- [Crafting Your Evolving Dreams: Concept-Incremental Versatile Customization](https://mlsanity.com/paper/2606.04797/) (2026)
- [Multi-view Action Recognition via Directed Gromov-Wasserstein Discrepancy](https://mlsanity.com/paper/2405.01337/) (2024)
- [BRAIN: Bias-Mitigation Continual Learning Approach to Vision-Brain Understanding](https://mlsanity.com/paper/2508.18187/) (2025)
- [FuTCR: Future-Targeted Contrast and Repulsion for Continual Panoptic Segmentation](https://mlsanity.com/paper/2605.12451/) (2026)
- [α-Graph: Attention-Infused Normalizing Flow Approach to Tractable Graph Modeling](https://mlsanity.com/paper/2609.07961/) (2026)

## Abstract (arXiv)

Continual Learning in semantic scene segmentation aims to continually learn new unseen classes in dynamic environments while maintaining previously learned knowledge. Prior studies focused on modeling the catastrophic forgetting and background shift challenges in continual learning. However, fairness, another major challenge that causes unfair predictions leading to low performance among major and minor classes, still needs to be well addressed. In addition, prior methods have yet to model the unknown classes well, thus resulting in producing non-discriminative features among unknown classes. This work presents a novel Fairness Learning via Contrastive Attention Approach to continual learning in semantic scene understanding. In particular, we first introduce a new Fairness Contrastive Clustering loss to address the problems of catastrophic forgetting and fairness. Then, we propose an attention-based visual grammar approach to effectively model the background shift problem and unknown classes, producing better feature representations for different unknown classes. Through our experiments, our proposed approach achieves State-of-the-Art (SoTA) performance on different continual learning benchmarks, i.e., ADE20K, Cityscapes, and Pascal VOC. It promotes the fairness of the continual semantic segmentation model.