# Approaching Peak Ground Truth

arXiv: https://arxiv.org/abs/2301.00243 · HTML: https://mlsanity.com/paper/2301.00243/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 39 references that are in the map.
- First posted on arXiv 31 Dec 2022.

## Builds on (most cited first, 19 of 19)

- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019)
- [Are we done with ImageNet?](https://mlsanity.com/paper/2006.07159/) (2020)
- [A Probabilistic U-Net for Segmentation of Ambiguous Images](https://mlsanity.com/paper/1806.05034/) (2018)
- [Pros and Cons of GAN Evaluation Measures](https://mlsanity.com/paper/1802.03446/) (2018)
- [Metrics reloaded: Recommendations for image analysis validation](https://mlsanity.com/paper/2206.01653/) (2022)
- [The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 Challenge](https://mlsanity.com/paper/1912.01054/) (2019)
- [A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis](https://mlsanity.com/paper/1910.02923/) (2019)
- [Pros and Cons of GAN Evaluation Measures: New Developments](https://mlsanity.com/paper/2103.09396/) (2021)
- [Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels](https://mlsanity.com/paper/2101.05022/) (2021)
- [Why rankings of biomedical image analysis competitions should be interpreted with care](https://mlsanity.com/paper/1806.02051/) (2018)
- [Common Limitations of Image Processing Metrics: A Picture Story](https://mlsanity.com/paper/2104.05642/) (2021)
- [Computing the Testing Error without a Testing Set](https://mlsanity.com/paper/2005.00450/) (2020)
- [An Unsupervised Information-Theoretic Perceptual Quality Metric](https://mlsanity.com/paper/2006.06752/) (2020)
- [Are we using appropriate segmentation metrics? Identifying correlates of human expert perception for CNN training beyond rolling the DICE coefficient](https://mlsanity.com/paper/2103.06205/) (2021)
- [Labeling instructions matter in biomedical image analysis](https://mlsanity.com/paper/2207.09899/) (2022)
- [Where is VALDO? VAscular Lesions Detection and segmentatiOn challenge at MICCAI 2021](https://mlsanity.com/paper/2208.07167/) (2022)
- [Deep Quality Estimation: Creating Surrogate Models for Human Quality Ratings](https://mlsanity.com/paper/2205.10355/) (2022)
- [Neural Annotation Refinement: Development of a New 3D Dataset for Adrenal Gland Analysis](https://mlsanity.com/paper/2206.15328/) (2022)

## Built on by (most cited first, 8 of 8)

- [QUBIQ: Uncertainty Quantification for Biomedical Image Segmentation Challenge](https://mlsanity.com/paper/2405.18435/) (2024)
- [Panoptica -- instance-wise evaluation of 3D semantic and instance segmentation maps](https://mlsanity.com/paper/2312.02608/) (2023)
- [The Brain Tumor Segmentation (BraTS) Challenge: Local Synthesis of Healthy Brain Tissue via Inpainting](https://mlsanity.com/paper/2305.08992/) (2023)
- [MultiOrg: A Multi-rater Organoid-detection Dataset](https://mlsanity.com/paper/2410.14612/) (2024)
- [Train-Free Segmentation in MRI with Cubical Persistent Homology](https://mlsanity.com/paper/2401.01160/) (2024)
- [BrainLesion Suite: A Flexible and User-Friendly Framework for Modular Brain Lesion Image Analysis](https://mlsanity.com/paper/2507.09036/) (2025)
- [Framing image registration as a landmark detection problem for label-noise-aware task representation (HitR)](https://mlsanity.com/paper/2308.01318/) (2023)
- [Enhancing efficiency in paediatric brain tumour segmentation using a pathologically diverse single-center clinical dataset](https://mlsanity.com/paper/2507.22152/) (2025)

## Abstract (arXiv)

Machine learning models are typically evaluated by computing similarity with reference annotations and trained by maximizing similarity with such. Especially in the biomedical domain, annotations are subjective and suffer from low inter- and intra-rater reliability. Since annotations only reflect one interpretation of the real world, this can lead to sub-optimal predictions even though the model achieves high similarity scores. Here, the theoretical concept of PGT is introduced. PGT marks the point beyond which an increase in similarity with the reference annotation stops translating to better RWMP. Additionally, a quantitative technique to approximate PGT by computing inter- and intra-rater reliability is proposed. Finally, four categories of PGT-aware strategies to evaluate and improve model performance are reviewed.