# Validity problems in clinical machine learning by indirect data labeling using consensus definitions

arXiv: https://arxiv.org/abs/2311.03037 · HTML: https://mlsanity.com/paper/2311.03037/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 22 references that are in the map.
- First posted on arXiv 6 Nov 2023.

## Builds on (most cited first, 5 of 5)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)
- [Neural Additive Models: Interpretable Machine Learning with Neural Nets](https://mlsanity.com/paper/2004.13912/) (2020)
- [Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation](https://mlsanity.com/paper/1710.06169/) (2017)
- [Ensembling Neural Networks for Improved Prediction and Privacy in Early Diagnosis of Sepsis](https://mlsanity.com/paper/2209.00439/) (2022)

## Built on by (most cited first, 2 of 2)

- [Statistically Valid Post-Deployment Monitoring Should Be Standard for AI-Based Digital Health](https://mlsanity.com/paper/2506.05701/) (2025)
- [Early Prediction of Causes (not Effects) in Healthcare by Long-Term Clinical Time Series Forecasting](https://mlsanity.com/paper/2408.03816/) (2024)

## Abstract (arXiv)

We demonstrate a validity problem of machine learning in the vital application area of disease diagnosis in medicine. It arises when target labels in training data are determined by an indirect measurement, and the fundamental measurements needed to determine this indirect measurement are included in the input data representation. Machine learning models trained on this data will learn nothing else but to exactly reconstruct the known target definition. Such models show perfect performance on similarly constructed test data but will fail catastrophically on real-world examples where the defining fundamental measurements are not or only incompletely available. We present a general procedure allowing identification of problematic datasets and black-box machine learning models trained on them, and exemplify our detection procedure on the task of early prediction of sepsis.