# Estimating label quality and errors in semantic segmentation data via any model

arXiv: https://arxiv.org/abs/2307.05080 · HTML: https://mlsanity.com/paper/2307.05080/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 30 references that are in the map.
- First posted on arXiv 11 Jul 2023.

## Builds on (most cited first, 22 of 22)

- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [Confident Learning: Estimating Uncertainty in Dataset Labels](https://mlsanity.com/paper/1911.00068/) (2019)
- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations](https://mlsanity.com/paper/2110.12088/) (2021)
- [Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels](https://mlsanity.com/paper/1905.05040/) (2019)
- [Entropy Maximization and Meta Classification for Out-Of-Distribution Detection in Semantic Segmentation](https://mlsanity.com/paper/2012.06575/) (2020)
- [Augmentation Strategies for Learning with Noisy Labels](https://mlsanity.com/paper/2103.02130/) (2021)
- [Disentangling Human Error from the Ground Truth in Segmentation of Medical Images](https://mlsanity.com/paper/2007.15963/) (2020)
- [Training Convolutional Networks with Noisy Labels](https://mlsanity.com/paper/1406.2080/) (2014)
- [Annotation Error Detection: Analyzing the Past and Present for a More Coherent Future](https://mlsanity.com/paper/2206.02280/) (2022)
- [Identifying Mislabeled Instances in Classification Datasets](https://mlsanity.com/paper/1912.05283/) (2019)
- [Automated Detection of Label Errors in Semantic Segmentation Datasets via Deep Learning and Uncertainty Quantification](https://mlsanity.com/paper/2207.06104/) (2022)
- [Imperfect Segmentation Labels: How Much Do They Matter?](https://mlsanity.com/paper/1806.04618/) (2018)
- [Detecting Label Errors in Token Classification Data](https://mlsanity.com/paper/2210.03920/) (2022)
- [Identifying Incorrect Annotations in Multi-Label Classification Data](https://mlsanity.com/paper/2211.13895/) (2022)
- [Combating noisy labels in object detection datasets](https://mlsanity.com/paper/2211.13993/) (2022)
- [ActiveLab: Active Learning with Re-Labeling by Multiple Annotators](https://mlsanity.com/paper/2301.11856/) (2023)

## Built on by (most cited first, 12 of 12)

- [Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning](https://mlsanity.com/paper/2502.01968/) (2025)
- [Active Label Correction for Semantic Segmentation with Foundation Models](https://mlsanity.com/paper/2403.10820/) (2024)
- [Adversarial Robustness of Deep Learning Models for Inland Water Body Segmentation from SAR Images](https://mlsanity.com/paper/2505.01884/) (2025)
- [A2LC: Active and Automated Label Correction for Semantic Segmentation](https://mlsanity.com/paper/2506.11599/) (2025)
- [Towards Fairness under Label Bias in Image Segmentation: Impact, Measurement and Mitigation](https://mlsanity.com/paper/2605.06891/) (2026)
- [Data-Centric Long-Tailed Image Recognition](https://mlsanity.com/paper/2311.01744/) (2023)
- [HyperSORT: Self-Organising Robust Training with hyper-networks](https://mlsanity.com/paper/2506.21430/) (2025)
- [Learning to Detect Label Errors by Making Them: A Method for Segmentation and Object Detection Datasets](https://mlsanity.com/paper/2508.17930/) (2025)
- [Ordinal Adaptive Correction: A Data-Centric Approach to Ordinal Image Classification with Noisy Labels](https://mlsanity.com/paper/2509.02351/) (2025)
- [Data-Centric Benchmark for Label Noise Estimation and Ranking in Remote Sensing Binary Building Segmentation](https://mlsanity.com/paper/2603.00604/) (2026)
- [Why Domain Matters: Domain-Aware Benchmarking of Underwater Object Detection and Annotation Quality](https://mlsanity.com/paper/2607.10575/) (2026)
- [Contrastive Mask Fidelity: Reference-Free Auditing of Ground-Truth Masks in Remote Sensing Semantic Segmentation](https://mlsanity.com/paper/2608.09101/) (2026)

## Abstract (arXiv)

The labor-intensive annotation process of semantic segmentation datasets is often prone to errors, since humans struggle to label every pixel correctly. We study algorithms to automatically detect such annotation errors, in particular methods to score label quality, such that the images with the lowest scores are least likely to be correctly labeled. This helps prioritize what data to review in order to ensure a high-quality training/evaluation dataset, which is critical in sensitive applications such as medical imaging and autonomous vehicles. Widely applicable, our label quality scores rely on probabilistic predictions from a trained segmentation model -- any model architecture and training procedure can be utilized. Here we study 7 different label quality scoring methods used in conjunction with a DeepLabV3+ or a FPN segmentation model to detect annotation errors in a version of the SYNTHIA dataset. Precision-recall evaluations reveal a score -- the soft-minimum of the model-estimated likelihoods of each pixel's annotated class -- that is particularly effective to identify images that are mislabeled, across multiple types of annotation error.