# Towards Fairness under Label Bias in Image Segmentation: Impact, Measurement and Mitigation

arXiv: https://arxiv.org/abs/2605.06891 · HTML: https://mlsanity.com/paper/2605.06891/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 80 references that are in the map.
- First posted on arXiv 7 May 2026.

## Builds on (most cited first, 30 of 41)

- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1710.05006/) (2017)
- [Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics](https://mlsanity.com/paper/2009.10795/) (2020)
- [Confident Learning: Estimating Uncertainty in Dataset Labels](https://mlsanity.com/paper/1911.00068/) (2019)
- [Training Deep Neural Networks on Noisy Labels with Bootstrapping](https://mlsanity.com/paper/1412.6596/) (2014)
- [Fairness without Demographics through Adversarially Reweighted Learning](https://mlsanity.com/paper/2006.13114/) (2020)
- [Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging](https://mlsanity.com/paper/1909.12475/) (2019)
- [Identifying and Correcting Label Bias in Machine Learning](https://mlsanity.com/paper/1901.04966/) (2019)
- [Fair Classification with Group-Dependent Label Noise](https://mlsanity.com/paper/2011.00379/) (2020)
- [Common Limitations of Image Processing Metrics: A Picture Story](https://mlsanity.com/paper/2104.05642/) (2021)
- [Disentangling Human Error from the Ground Truth in Segmentation of Medical Images](https://mlsanity.com/paper/2007.15963/) (2020)
- [Leveling Down in Computer Vision: Pareto Inefficiencies in Fair Deep Classifiers](https://mlsanity.com/paper/2203.04913/) (2022)
- [On the Effect of Inter-observer Variability for a Reliable Estimation of Uncertainty of Medical Image Segmentation](https://mlsanity.com/paper/1806.02562/) (2018)
- [Consensus and Subjectivity of Skin Tone Annotation for ML Fairness](https://mlsanity.com/paper/2305.09073/) (2023)
- [Learning to Segment from Noisy Annotations: A Spatial Correction Approach](https://mlsanity.com/paper/2308.02498/) (2023)
- [Imperfect Segmentation Labels: How Much Do They Matter?](https://mlsanity.com/paper/1806.04618/) (2018)
- [That Label's Got Style: Handling Label Style Bias for Uncertain Image Segmentation](https://mlsanity.com/paper/2303.15850/) (2023)
- [Do We Really Need Dice? The Hidden Region-Size Biases of Segmentation Losses](https://mlsanity.com/paper/2104.08717/) (2021)
- [Estimating label quality and errors in semantic segmentation data via any model](https://mlsanity.com/paper/2307.05080/) (2023)

## Built on by (most cited first, 2 of 2)

- [False Confidence: Automated Labels Confound Fairness Audits in Cervical Spine Segmentation](https://mlsanity.com/paper/2607.07852/) (2026)
- [Is invariance all you need for algorithmic fairness? Removing demographic information can create new bias](https://mlsanity.com/paper/2609.32004/) (2026)

## Abstract (arXiv)

Biases in annotation pipelines may introduce label bias: when annotation quality or style varies systematically across demographic subgroups, causing disparities in performance and auditability. Label bias in image segmentation remains underexplored, as even detecting it typically requires clean, unbiased annotations, which are not readily available. We present a Confident Learning framework adapted to segmentation for auditing label bias directly in the training data without a clean, unbiased ground truth. By comparing the provided training labels to the model's confident predictions, we measure the prevalence and direction of suspected label errors; where standard overlap metrics like Dice fail. We further show that label bias influences subgroup separability in the encoder's feature space, an artifact we leverage for bias mitigation rather than suppressing it. We evaluate three datasets spanning from synthetic to real-life bias and across experimental conditions, showing how our framework produces proxy audit signals without clean labels and mitigates disparities with a correctly identified cleaner reference group.