# Improving Object Detection in Medical Image Analysis through Multiple Expert Annotators: An Empirical Investigation

arXiv: https://arxiv.org/abs/2303.16507 · HTML: https://mlsanity.com/paper/2303.16507/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 37 references that are in the map.
- First posted on arXiv 29 Mar 2023.

## Builds on (most cited first, 18 of 18)

- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Object Detection in 20 Years: A Survey](https://mlsanity.com/paper/1905.05055/) (2019)
- [VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations](https://mlsanity.com/paper/2012.15029/) (2020)
- [Learning From Noisy Labels By Regularized Estimation Of Annotator Confusion](https://mlsanity.com/paper/1902.03680/) (2019)
- [Deep learning from crowds](https://mlsanity.com/paper/1709.01779/) (2017)
- [VinDr-Mammo: A large-scale benchmark dataset for computer-aided diagnosis in full-field digital mammography](https://mlsanity.com/paper/2203.11205/) (2022)
- [Disentangling Human Error from the Ground Truth in Segmentation of Medical Images](https://mlsanity.com/paper/2007.15963/) (2020)
- [Interpreting Chest X-rays via CNNs that Exploit Hierarchical Disease Dependencies and Uncertainty Labels](https://mlsanity.com/paper/2005.12734/) (2020)
- [VinDr-SpineXR: A deep learning framework for spinal lesions detection and classification from radiographs](https://mlsanity.com/paper/2106.12930/) (2021)
- [Learning from Multiple Expert Annotators for Enhancing Anomaly Detection in Medical Image Analysis](https://mlsanity.com/paper/2203.10611/) (2022)
- [DICOM Imaging Router: An Open Deep Learning Framework for Classification of Body Parts from DICOM X-ray Scans](https://mlsanity.com/paper/2108.06490/) (2021)
- [A Technical Survey on Statistical Modelling and Design Methods for Crowdsourcing Quality Control](https://mlsanity.com/paper/1812.02736/) (2018)
- [Learning to Automatically Diagnose Multiple Diseases in Pediatric Chest Radiographs Using Deep Convolutional Neural Networks](https://mlsanity.com/paper/2108.06486/) (2021)
- [An Accurate and Explainable Deep Learning System Improves Interobserver Agreement in the Interpretation of Chest Radiograph](https://mlsanity.com/paper/2208.03545/) (2022)
- [Learning to diagnose common thorax diseases on chest radiographs from radiology reports in Vietnamese](https://mlsanity.com/paper/2209.04794/) (2022)
- [A clinical validation of VinDr-CXR, an AI system for detecting abnormal chest radiographs](https://mlsanity.com/paper/2104.02256/) (2021)
- [Learning from Multiple Annotators by Incorporating Instance Features](https://mlsanity.com/paper/2106.15146/) (2021)

## Built on by (most cited first, 1 of 1)

- [Label Convergence: Defining an Upper Performance Bound in Object Recognition through Contradictory Annotations](https://mlsanity.com/paper/2409.09412/) (2024)

## Abstract (arXiv)

The work discusses the use of machine learning algorithms for anomaly detection in medical image analysis and how the performance of these algorithms depends on the number of annotators and the quality of labels. To address the issue of subjectivity in labeling with a single annotator, we introduce a simple and effective approach that aggregates annotations from multiple annotators with varying levels of expertise. We then aim to improve the efficiency of predictive models in abnormal detection tasks by estimating hidden labels from multiple annotations and using a re-weighted loss function to improve detection performance. Our method is evaluated on a real-world medical imaging dataset and outperforms relevant baselines that do not consider disagreements among annotators.