# Label Convergence: Defining an Upper Performance Bound in Object Recognition through Contradictory Annotations

arXiv: https://arxiv.org/abs/2409.09412 · HTML: https://mlsanity.com/paper/2409.09412/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 47 references that are in the map.
- First posted on arXiv 14 Sep 2024.

## Builds on (most cited first, 26 of 26)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [DETRs with Collaborative Hybrid Assignments Training](https://mlsanity.com/paper/2211.12860/) (2022)
- [VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations](https://mlsanity.com/paper/2012.15029/) (2020)
- [The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation](https://mlsanity.com/paper/2211.02570/) (2022)
- [Learning From Noisy Labels By Regularized Estimation Of Annotator Confusion](https://mlsanity.com/paper/1902.03680/) (2019)
- [Learning from Noisy Anchors for One-stage Object Detection](https://mlsanity.com/paper/1912.05086/) (2019)
- [Large-Scale Object Detection in the Wild from Imbalanced Multi-Labels](https://mlsanity.com/paper/2005.08455/) (2020)
- [Towards Noise-resistant Object Detection with Noisy Annotations](https://mlsanity.com/paper/2003.01285/) (2020)
- [Robust Object Detection With Inaccurate Bounding Boxes](https://mlsanity.com/paper/2207.09697/) (2022)
- [Training Object Detectors With Noisy Data](https://mlsanity.com/paper/1905.07202/) (2019)
- [Identifying Label Errors in Object Detection Datasets by Loss Inspection](https://mlsanity.com/paper/2303.06999/) (2023)
- [Learning from Multiple Expert Annotators for Enhancing Anomaly Detection in Medical Image Analysis](https://mlsanity.com/paper/2203.10611/) (2022)
- [Empirical Upper Bound in Object Detection and More](https://mlsanity.com/paper/1911.12451/) (2019)
- [Don't Blame the Data, Blame the Model: Understanding Noise and Bias When Learning from Subjective Annotations](https://mlsanity.com/paper/2403.04085/) (2024)
- [Handling Inter-Annotator Agreement for Automated Skin Lesion Segmentation](https://mlsanity.com/paper/1906.02415/) (2019)
- [Noisy Annotation Refinement for Object Detection](https://mlsanity.com/paper/2110.10456/) (2021)
- [Assessing Data Quality of Annotations with Krippendorff Alpha For Applications in Computer Vision](https://mlsanity.com/paper/1912.10107/) (2019)
- [An Accurate and Explainable Deep Learning System Improves Interobserver Agreement in the Interpretation of Chest Radiograph](https://mlsanity.com/paper/2208.03545/) (2022)
- [Drawing the Same Bounding Box Twice? Coping Noisy Annotations in Object Detection with Repeated Labels](https://mlsanity.com/paper/2309.09742/) (2023)
- [Improving Object Detection in Medical Image Analysis through Multiple Expert Annotators: An Empirical Investigation](https://mlsanity.com/paper/2303.16507/) (2023)
- [NoiseBench: Benchmarking the Impact of Real Label Noise on Named Entity Recognition](https://mlsanity.com/paper/2405.07609/) (2024)

## Built on by (most cited first, 6 of 6)

- [TMI: Text-to-Image Meets Image-to-Image for Complementary Data Synthesis to Boost Long-Tailed Instance Segmentation](https://mlsanity.com/paper/2607.08201/) (2026)
- [CISOL: An Open and Extensible Dataset for Table Structure Recognition in the Construction Industry](https://mlsanity.com/paper/2501.15469/) (2025)
- [IndoorCrowd: A Multi-Scene Dataset for Human Detection, Segmentation, and Tracking with an Automated Annotation Pipeline](https://mlsanity.com/paper/2604.02032/) (2026)
- [Calibrating Probabilistic Object Detectors with Annotator Disagreement](https://mlsanity.com/paper/2605.24722/) (2026)
- [Analyzing Training-Free Corruption Detection for Object Detection Datasets](https://mlsanity.com/paper/2606.10666/) (2026)
- [Quantifying Annotation Agreement in Object Detection with Optimal Transport](https://mlsanity.com/paper/2610.19912/) (2026)

## Abstract (arXiv)

Annotation errors are a challenge not only during training of machine learning models, but also during their evaluation. Label variations and inaccuracies in datasets often manifest as contradictory examples that deviate from established labeling conventions. Such inconsistencies, when significant, prevent models from achieving optimal performance on metrics such as mean Average Precision (mAP). We introduce the notion of "label convergence" to describe the highest achievable performance under the constraint of contradictory test annotations, essentially defining an upper bound on model accuracy. Recognizing that noise is an inherent characteristic of all data, our study analyzes five real-world datasets, including the LVIS dataset, to investigate the phenomenon of label convergence. We approximate that label convergence is between 62.63-67.52 mAP@[0.5:0.95:0.05] for LVIS with 95% confidence, attributing these bounds to the presence of real annotation errors. With current state-of-the-art (SOTA) models at the upper end of the label convergence interval for the well-studied LVIS dataset, we conclude that model capacity is sufficient to solve current object detection problems. Therefore, future efforts should focus on three key aspects: (1) updating the problem specification and adjusting evaluation practices to account for unavoidable label noise, (2) creating cleaner data, especially test data, and (3) including multi-annotated data to investigate annotation variation and make these issues visible from the outset.