# Are pathologist-defined labels reproducible? Comparison of the TUPAC16 mitotic figure dataset with an alternative set of labels

arXiv: https://arxiv.org/abs/2007.05351 · HTML: https://mlsanity.com/paper/2007.05351/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jul 2020.

## Built on by (most cited first, 12 of 12)

- [Mitosis domain generalization in histopathology images -- The MIDOG challenge](https://mlsanity.com/paper/2204.03742/) (2022)
- [Domain generalization across tumor types, laboratories, and species -- insights from the 2022 edition of the Mitosis Domain Generalization Challenge](https://mlsanity.com/paper/2309.15589/) (2023)
- [Domain Adversarial RetinaNet as a Reference Algorithm for the MItosis DOmain Generalization Challenge](https://mlsanity.com/paper/2108.11269/) (2021)
- [A Systematic Review of Generalization Research in Medical Image Classification](https://mlsanity.com/paper/2403.12167/) (2024)
- [Benchmarking Deep Learning and Vision Foundation Models for Atypical vs. Normal Mitosis Classification with Cross-Dataset Evaluation](https://mlsanity.com/paper/2506.21444/) (2025)
- [Mitosis Detection, Fast and Slow: Robust and Efficient Detection of Mitotic Figures](https://mlsanity.com/paper/2208.12587/) (2022)
- [An Efficient Cervical Whole Slide Image Analysis Framework Based on Multi-scale Semantic and Location Deep Features](https://mlsanity.com/paper/2106.15113/) (2021)
- [Automatic and explainable grading of meningiomas from histopathology images](https://mlsanity.com/paper/2107.08850/) (2021)
- [Positive-unlabeled learning for binary and multi-class cell detection in histopathology images with incomplete annotations](https://mlsanity.com/paper/2302.08050/) (2023)
- [Characterizing the Features of Mitotic Figures Using a Conditional Diffusion Probabilistic Model](https://mlsanity.com/paper/2310.03893/) (2023)
- [On the Value of PHH3 for Mitotic Figure Detection on H&E-stained Images](https://mlsanity.com/paper/2406.19899/) (2024)
- [Beyond Classification: Pathology Foundation Models as Detection Encoders for Mitotic Figures](https://mlsanity.com/paper/2607.28007/) (2026)

## Abstract (arXiv)

Pathologist-defined labels are the gold standard for histopathological data sets, regardless of well-known limitations in consistency for some tasks. To date, some datasets on mitotic figures are available and were used for development of promising deep learning-based algorithms. In order to assess robustness of those algorithms and reproducibility of their methods it is necessary to test on several independent datasets. The influence of different labeling methods of these available datasets is currently unknown. To tackle this, we present an alternative set of labels for the images of the auxiliary mitosis dataset of the TUPAC16 challenge. Additional to manual mitotic figure screening, we used a novel, algorithm-aided labeling process, that allowed to minimize the risk of missing rare mitotic figures in the images. All potential mitotic figures were independently assessed by two pathologists. The novel, publicly available set of labels contains 1,999 mitotic figures (+28.80%) and additionally includes 10,483 labels of cells with high similarities to mitotic figures (hard examples). We found significant difference comparing F_1 scores between the original label set (0.549) and the new alternative label set (0.735) using a standard deep learning object detection architecture. The models trained on the alternative set showed higher overall confidence values, suggesting a higher overall label consistency. Findings of the present study show that pathologists-defined labels may vary significantly resulting in notable difference in the model performance. Comparison of deep learning-based algorithms between independent datasets with different labeling methods should be done with caution.