# Semantic Segmentation with Labeling Uncertainty and Class Imbalance

arXiv: https://arxiv.org/abs/2102.04566 · HTML: https://mlsanity.com/paper/2102.04566/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 42 references that are in the map.
- First posted on arXiv 8 Feb 2021.

## Builds on (most cited first, 16 of 16)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets](https://mlsanity.com/paper/1707.08819/) (2017)
- [An open access repository of images on plant health to enable the development of mobile disease diagnostics](https://mlsanity.com/paper/1511.08060/) (2015)
- [Boundary-Aware Feature Propagation for Scene Segmentation](https://mlsanity.com/paper/1909.00179/) (2019)
- [Imbalanced Deep Learning by Minority Class Incremental Rectification](https://mlsanity.com/paper/1804.10851/) (2018)
- [Loss Max-Pooling for Semantic Image Segmentation](https://mlsanity.com/paper/1704.02966/) (2017)
- [Effective Use of Dilated Convolutions for Segmenting Small Object Instances in Remote Sensing Imagery](https://mlsanity.com/paper/1709.00179/) (2017)
- [Label Refinement Network for Coarse-to-Fine Semantic Segmentation](https://mlsanity.com/paper/1703.00551/) (2017)
- [High-performance Semantic Segmentation Using Very Deep Fully Convolutional Networks](https://mlsanity.com/paper/1604.04339/) (2016)
- [PixelNet: Towards a General Pixel-level Architecture](https://mlsanity.com/paper/1609.06694/) (2016)
- [Application of Decision Rules for Handling Class Imbalance in Semantic Segmentation](https://mlsanity.com/paper/1901.08394/) (2019)
- [Joint Calibration for Semantic Segmentation](https://mlsanity.com/paper/1507.01581/) (2015)

## Built on by (most cited first, 12 of 12)

- [Lightweight, Pre-trained Transformers for Remote Sensing Timeseries](https://mlsanity.com/paper/2304.14065/) (2023)
- [The Segment Anything Model (SAM) for Remote Sensing Applications: From Zero to One Shot](https://mlsanity.com/paper/2306.16623/) (2023)
- [A Simple yet Effective Network based on Vision Transformer for Camouflaged Object and Salient Object Detection](https://mlsanity.com/paper/2402.18922/) (2024)
- [Hyperbolic Uncertainty Aware Semantic Segmentation](https://mlsanity.com/paper/2203.08881/) (2022)
- [CFPNet-M: A Light-Weight Encoder-Decoder Based Network for Multimodal Biomedical Image Real-Time Segmentation](https://mlsanity.com/paper/2105.04075/) (2021)
- [U-CE: Uncertainty-aware Cross-Entropy for Semantic Segmentation](https://mlsanity.com/paper/2307.09947/) (2023)
- [AerOSeg: Harnessing SAM for Open-Vocabulary Segmentation in Remote Sensing Images](https://mlsanity.com/paper/2504.09203/) (2025)
- [Compensation Learning in Semantic Segmentation](https://mlsanity.com/paper/2304.13428/) (2023)
- [Beyond Nearest Neighbor Interpolation in Data Augmentation](https://mlsanity.com/paper/2504.01527/) (2025)
- [Label Adversarial Learning for Skeleton-level to Pixel-level Adjustable Vessel Segmentation](https://mlsanity.com/paper/2205.03646/) (2022)
- [Instance-wise Uncertainty for Class Imbalance in Semantic Segmentation](https://mlsanity.com/paper/2407.12609/) (2024)
- [Transport-Related Surface Detection with Machine Learning: Analyzing Temporal Trends in Madrid and Vienna](https://mlsanity.com/paper/2503.15653/) (2025)

## Abstract (arXiv)

Recently, methods based on Convolutional Neural Networks (CNN) achieved impressive success in semantic segmentation tasks. However, challenges such as the class imbalance and the uncertainty in the pixel-labeling process are not completely addressed. As such, we present a new approach that calculates a weight for each pixel considering its class and uncertainty during the labeling process. The pixel-wise weights are used during training to increase or decrease the importance of the pixels. Experimental results show that the proposed approach leads to significant improvements in three challenging segmentation tasks in comparison to baseline methods. It was also proved to be more invariant to noise. The approach presented here may be used within a wide range of semantic segmentation methods to improve their robustness.