# Annotating Ambiguous Images: General Annotation Strategy for High-Quality Data with Real-World Biomedical Validation

arXiv: https://arxiv.org/abs/2306.12189 · HTML: https://mlsanity.com/paper/2306.12189/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 62 references that are in the map.
- First posted on arXiv 21 Jun 2023.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Are we done with ImageNet?](https://mlsanity.com/paper/2006.07159/) (2020)
- [Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](https://mlsanity.com/paper/2110.05719/) (2021)
- [Human uncertainty makes classification more robust](https://mlsanity.com/paper/1908.07086/) (2019)
- [Does label smoothing mitigate label noise?](https://mlsanity.com/paper/2003.02819/) (2020)
- [Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels](https://mlsanity.com/paper/2101.05022/) (2021)
- [Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspective](https://mlsanity.com/paper/2112.06409/) (2021)
- [VerSe: A Vertebrae Labelling and Segmentation Benchmark for Multi-detector CT Images](https://mlsanity.com/paper/2001.09193/) (2020)
- [OmniMAE: Single Model Masked Pretraining on Images and Videos](https://mlsanity.com/paper/2206.08356/) (2022)
- [When does dough become a bagel? Analyzing the remaining mistakes on ImageNet](https://mlsanity.com/paper/2205.04596/) (2022)
- [Learning Customized Visual Models with Retrieval-Augmented Knowledge](https://mlsanity.com/paper/2301.07094/) (2023)
- [A Realistic Fish-Habitat Dataset to Evaluate Algorithms for Underwater Visual Analysis](https://mlsanity.com/paper/2008.12603/) (2020)
- [Is one annotation enough? A data-centric image classification benchmark for noisy and ambiguous label estimation](https://mlsanity.com/paper/2207.06214/) (2022)
- [Isometric Propagation Network for Generalized Zero-shot Learning](https://mlsanity.com/paper/2102.02038/) (2021)
- [A survey of image labelling for computer vision applications](https://mlsanity.com/paper/2104.08885/) (2021)
- [Towards Good Practices for Efficiently Annotating Large-Scale Image Classification Datasets](https://mlsanity.com/paper/2104.12690/) (2021)
- [Beyond Cats and Dogs: Semi-supervised Classification of fuzzy labels with overclustering](https://mlsanity.com/paper/2012.01768/) (2020)
- [Adaptive Label Smoothing](https://mlsanity.com/paper/2009.06432/) (2020)
- [MorphoCluster: Efficient Annotation of Plankton images by Clustering](https://mlsanity.com/paper/2005.01595/) (2020)
- [Deep learning with self-supervision and uncertainty regularization to count fish in underwater images](https://mlsanity.com/paper/2104.14964/) (2021)
- [A data-centric approach for improving ambiguous labels with combined semi-supervised classification and clustering](https://mlsanity.com/paper/2106.16209/) (2021)
- [Scaling up instance annotation via label propagation](https://mlsanity.com/paper/2110.02277/) (2021)

## Built on by (most cited first, 1 of 1)

- [A Probabilistic Segment Anything Model for Ambiguity-Aware Medical Image Segmentation](https://mlsanity.com/paper/2509.05809/) (2025)

## Abstract (arXiv)

In the field of image classification, existing methods often struggle with biased or ambiguous data, a prevalent issue in real-world scenarios. Current strategies, including semi-supervised learning and class blending, offer partial solutions but lack a definitive resolution. Addressing this gap, our paper introduces a novel strategy for generating high-quality labels in challenging datasets. Central to our approach is a clearly designed flowchart, based on a broad literature review, which enables the creation of reliable labels. We validate our methodology through a rigorous real-world test case in the biomedical field, specifically in deducing height reduction from vertebral imaging. Our empirical study, leveraging over 250,000 annotations, demonstrates the effectiveness of our strategies decisions compared to their alternatives.