A Multi-Annotator Study of Segmentation Noise and Uncertainty in Turbid Underwater Images
Not on the map. Why?
Abstract (arXiv)
Label uncertainty and annotator disagreement are common challenges in the field of computer vision, yet their study has largely been confined to the medical domain or to generic image-recognition datasets. Underwater datasets are particularly susceptible to these issues due to the need for domain expertise, degraded visibility conditions, and the inherent difficulty of establishing reliable ground truth in inaccessible environments. Despite these challenges, annotation uncertainty in underwater imagery remains largely unexplored. In this work, we present the first systematic multi-annotator study of segmentation in real underwater scenes, with over 100 participants, and across varying, controlled levels of turbidity. We show that underwater datasets face many of the same annotation challenges as other vision tasks, while turbidity introduces additional systematic errors. We further investigate the main factors driving label noise and explore ways to improve annotation quality in turbid underwater environments, including privileged information, individual effort and annotator ensembles. All (meta-) data collected in this study will be available on the project page: https://vap.aau.dk/tubcertainty
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 61 references that are in the map. First posted on arXiv 15 Aug 2026.
Builds on
24 of 61 references.
References · 24
- 2017Semantic Foggy Scene Understanding with Synthetic Datacited by 403
- 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 395
- 2020Semantic Segmentation of Underwater Imagery: Dataset and Benchmarkcited by 95
- 2020From ImageNet to Image Classification: Contextualizing Progress on Benchmarkscited by 90
- 2020Map-Guided Curriculum Domain Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentationcited by 86
- 2022Is one annotation enough? A data-centric image classification benchmark for noisy and ambiguous label estimationcited by 43
- 2024VariErr NLI: Separating Annotation Error from Human Label Variationcited by 42
- 2023The Treasure Beneath Multiple Annotations: An Uncertainty-aware Edge Detectorcited by 25
- 2023Consensus and Subjectivity of Skin Tone Annotation for ML Fairnesscited by 22
- 2022Automated Detection of Label Errors in Semantic Segmentation Datasets via Deep Learning and Uncertainty Quantificationcited by 20
- 2022Labeling instructions matter in biomedical image analysiscited by 20
- 2013An Empirical Study into Annotator Agreement, Ground Truth Estimation, and Algorithm Evaluationcited by 16
12 more
- 2025Exploring the Underwater World Segmentation without Extra Trainingcited by 16
- 2023That Label's Got Style: Handling Label Style Bias for Uncertain Image Segmentationcited by 13
- 2019Handling Inter-Annotator Agreement for Automated Skin Lesion Segmentationcited by 9
- 2025The Coralscapes Dataset: Semantic Scene Understanding in Coral Reefscited by 9
- 2023Multi-annotator Deep Learning: A Probabilistic Framework for Classificationcited by 8
- 2023How to Efficiently Annotate Images for Best-Performing Deep Learning Based Segmentation Models: An Empirical Study with Weak and Noisy Annotations and Segment Anything Modelcited by 7
- 2024Arctique: An artificial histopathological dataset unifying realism and controllability for uncertainty quantificationcited by 4
- 2025AI-Driven Marine Robotics: Emerging Trends in Underwater Perception and Ecosystem Monitoringcited by 3
- 2024Performance of Human Annotators in Object Detection and Segmentation of Remotely Sensed Datacited by 2
- 2025What Can We Learn from Inter-Annotator Variability in Skin Lesion Segmentation?cited by 2
- 2024SeagrassFinder: Deep Learning for Eelgrass Detection and Coverage Estimation in the Wildcited by 1
- 2026Beyond Aesthetics: Quantifying Information Loss in Turbid Scenescited by 1
Built on by
Not cited yet.