# Reliable Benchmarking of Artifact Detection in Computational Pathology: A Reproducibility and Uncertainty Analysis

arXiv: https://arxiv.org/abs/2608.30835 · HTML: https://mlsanity.com/paper/2608.30835/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 48 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Builds on (most cited first, 12 of 12)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Accounting for Variance in Machine Learning Benchmarks](https://mlsanity.com/paper/2103.03098/) (2021)
- [Denoising diffusion models for out-of-distribution detection](https://mlsanity.com/paper/2211.07740/) (2022)
- [A Multi-resolution Model for Histopathology Image Classification and Localization with Multiple Instance Learning](https://mlsanity.com/paper/2011.02679/) (2020)
- [Efficient Quality Control of Whole Slide Pathology Images with Human-in-the-loop Training](https://mlsanity.com/paper/2409.19587/) (2024)
- [Equipping Computational Pathology Systems with Artifact Processing Pipelines: A Showcase for Computation and Performance Trade-offs](https://mlsanity.com/paper/2403.07743/) (2024)
- [PixCell: A generative foundation model for digital histopathology images](https://mlsanity.com/paper/2506.05127/) (2025)
- [FAST: A Dual-tier Few-Shot Learning Paradigm for Whole Slide Image Classification](https://mlsanity.com/paper/2409.19720/) (2024)
- [Reproducibility in machine learning for medical imaging](https://mlsanity.com/paper/2209.05097/) (2022)
- [Rapid Artefact Removal and H&E-Stained Tissue Segmentation](https://mlsanity.com/paper/2308.13304/) (2023)
- [Integration of nested cross-validation, automated hyperparameter optimization, high-performance computing to reduce and quantify the variance of test performance estimation of deep learning models](https://mlsanity.com/paper/2503.08589/) (2025)

## Built on by (most cited first, 1 of 1)

- [The Effect of Tissue Detection on False Positives of Diffusion-Based Artifact Detection in Histopathology](https://mlsanity.com/paper/2609.40083/) (2026)

## Abstract (arXiv)

Background and Objective: Quality control is a prerequisite for whole-slide image analysis, yet the benchmarks on which quality-control methods are compared share four properties that make their reported differences hard to interpret: few independent slides, annotation concentrated in a minority of them, pooled ratio metrics with no closed-form standard error, and a single inherited train/test partition. We propose a reliability protocol for such benchmarks. Methods: The protocol quantifies four sources of variability - test-set sampling, training stochasticity, partition composition, and undocumented preprocessing - a claim is reportable only if it survives all four; three of the four cost minutes of compute. We apply it to an independent reconstruction of a published diffusion-based artifact detector, evaluated on the original 24-slide partition and against a supervised baseline. Results: The method's central mechanism reproduces: the auxiliary contrastive term improves pooled F1 from 0.673 to 0.688 and replicates under a second seed (+0.0156, p = 0.031; +0.0190, p = 0.005), although it acts on pen marking rather than the artifact types cited to motivate it. Its comparative claims do not: differences between design variants, and against the supervised baseline, fall inside the uncertainty of the evaluation. Four of 24 slides carry 70% of scored annotated pixels, giving an effective sample size of 6.2, and the inherited partition sits at the 7th percentile. An unreported tissue-restriction step excludes 41.4% of out-of-focus annotation against 2.6% of air bubble; such a gate is confounded with blur by construction. Conclusions: Small-cohort benchmarks support far weaker conclusions than current reporting implies. The four checks are cheap enough to accompany any evaluation on such a resource and separate reproducible effects from differences the evaluation cannot resolve.