# A Multi-Annotator Study of Segmentation Noise and Uncertainty in Turbid Underwater Images

arXiv: https://arxiv.org/abs/2608.15363 · HTML: https://mlsanity.com/paper/2608.15363/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 61 references that are in the map.
- First posted on arXiv 15 Aug 2026.

## Builds on (most cited first, 24 of 24)

- [Semantic Foggy Scene Understanding with Synthetic Data](https://mlsanity.com/paper/1708.07819/) (2017)
- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [Semantic Segmentation of Underwater Imagery: Dataset and Benchmark](https://mlsanity.com/paper/2004.01241/) (2020)
- [From ImageNet to Image Classification: Contextualizing Progress on Benchmarks](https://mlsanity.com/paper/2005.11295/) (2020)
- [Map-Guided Curriculum Domain Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation](https://mlsanity.com/paper/2005.14553/) (2020)
- [Is one annotation enough? A data-centric image classification benchmark for noisy and ambiguous label estimation](https://mlsanity.com/paper/2207.06214/) (2022)
- [VariErr NLI: Separating Annotation Error from Human Label Variation](https://mlsanity.com/paper/2403.01931/) (2024)
- [The Treasure Beneath Multiple Annotations: An Uncertainty-aware Edge Detector](https://mlsanity.com/paper/2303.11828/) (2023)
- [Consensus and Subjectivity of Skin Tone Annotation for ML Fairness](https://mlsanity.com/paper/2305.09073/) (2023)
- [Automated Detection of Label Errors in Semantic Segmentation Datasets via Deep Learning and Uncertainty Quantification](https://mlsanity.com/paper/2207.06104/) (2022)
- [Labeling instructions matter in biomedical image analysis](https://mlsanity.com/paper/2207.09899/) (2022)
- [An Empirical Study into Annotator Agreement, Ground Truth Estimation, and Algorithm Evaluation](https://mlsanity.com/paper/1307.0426/) (2013)
- [Exploring the Underwater World Segmentation without Extra Training](https://mlsanity.com/paper/2511.07923/) (2025)
- [That Label's Got Style: Handling Label Style Bias for Uncertain Image Segmentation](https://mlsanity.com/paper/2303.15850/) (2023)
- [Handling Inter-Annotator Agreement for Automated Skin Lesion Segmentation](https://mlsanity.com/paper/1906.02415/) (2019)
- [The Coralscapes Dataset: Semantic Scene Understanding in Coral Reefs](https://mlsanity.com/paper/2503.20000/) (2025)
- [Multi-annotator Deep Learning: A Probabilistic Framework for Classification](https://mlsanity.com/paper/2304.02539/) (2023)
- [How to Efficiently Annotate Images for Best-Performing Deep Learning Based Segmentation Models: An Empirical Study with Weak and Noisy Annotations and Segment Anything Model](https://mlsanity.com/paper/2312.10600/) (2023)
- [Arctique: An artificial histopathological dataset unifying realism and controllability for uncertainty quantification](https://mlsanity.com/paper/2411.07097/) (2024)
- [AI-Driven Marine Robotics: Emerging Trends in Underwater Perception and Ecosystem Monitoring](https://mlsanity.com/paper/2509.01878/) (2025)
- [Performance of Human Annotators in Object Detection and Segmentation of Remotely Sensed Data](https://mlsanity.com/paper/2409.10272/) (2024)
- [What Can We Learn from Inter-Annotator Variability in Skin Lesion Segmentation?](https://mlsanity.com/paper/2508.09381/) (2025)
- [SeagrassFinder: Deep Learning for Eelgrass Detection and Coverage Estimation in the Wild](https://mlsanity.com/paper/2412.16147/) (2024)
- [Beyond Aesthetics: Quantifying Information Loss in Turbid Scenes](https://mlsanity.com/paper/2606.26295/) (2026)

## Abstract (arXiv)

Label uncertainty and annotator disagreement are common challenges in the field of computer vision, yet their study has largely been confined to the medical domain or to generic image-recognition datasets. Underwater datasets are particularly susceptible to these issues due to the need for domain expertise, degraded visibility conditions, and the inherent difficulty of establishing reliable ground truth in inaccessible environments. Despite these challenges, annotation uncertainty in underwater imagery remains largely unexplored. In this work, we present the first systematic multi-annotator study of segmentation in real underwater scenes, with over 100 participants, and across varying, controlled levels of turbidity. We show that underwater datasets face many of the same annotation challenges as other vision tasks, while turbidity introduces additional systematic errors. We further investigate the main factors driving label noise and explore ways to improve annotation quality in turbid underwater environments, including privileged information, individual effort and annotator ensembles. All (meta-) data collected in this study will be available on the project page: https://vap.aau.dk/tubcertainty