# On Leakage in Machine Learning Pipelines

arXiv: https://arxiv.org/abs/2311.04179 · HTML: https://mlsanity.com/paper/2311.04179/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 101 references that are in the map.
- First posted on arXiv 7 Nov 2023.

## Builds on (most cited first, 15 of 15)

- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice](https://mlsanity.com/paper/2007.15745/) (2020)
- [Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learning](https://mlsanity.com/paper/2007.04074/) (2020)
- [Challenges in Deploying Machine Learning: a Survey of Case Studies](https://mlsanity.com/paper/2011.09926/) (2020)
- [The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards](https://mlsanity.com/paper/1805.03677/) (2018)
- [Bias in Machine Learning Software: Why? How? What to do?](https://mlsanity.com/paper/2105.12195/) (2021)
- [Cross-validation: what does it estimate and how well does it do it?](https://mlsanity.com/paper/2104.00673/) (2021)
- [Leakage and the Reproducibility Crisis in ML-based Science](https://mlsanity.com/paper/2207.07048/) (2022)
- [Evaluating time series forecasting models: An empirical study on performance estimation methods](https://mlsanity.com/paper/1905.11744/) (2019)
- [Deep Learning Predicts Hip Fracture using Confounding Patient and Healthcare Variables](https://mlsanity.com/paper/1811.03695/) (2018)
- [Preventing False Discovery in Interactive Data Analysis is Hard](https://mlsanity.com/paper/1408.1655/) (2014)
- [Assessing and tuning brain decoders: cross-validation, caveats, and guidelines](https://mlsanity.com/paper/1606.05201/) (2016)
- [A Too-Good-to-be-True Prior to Reduce Shortcut Reliance](https://mlsanity.com/paper/2102.06406/) (2021)
- [Confound-leakage: Confound Removal in Machine Learning Leads to Leakage](https://mlsanity.com/paper/2210.09232/) (2022)

## Built on by (most cited first, 7 of 7)

- [Radiomics in Medical Imaging: Methods, Applications, and Challenges](https://mlsanity.com/paper/2602.00102/) (2026)
- [Outlier Detection in Plantar Pressure: Human-Centered Comparison of Statistical Parametric Mapping and Explainable Machine Learning](https://mlsanity.com/paper/2509.21943/) (2025)
- [ML2B: Benchmarking LLMs on Cross-Lingual ML Pipeline Generation](https://mlsanity.com/paper/2509.22768/) (2025)
- [A geometric and deep learning reproducible pipeline for monitoring floating anthropogenic debris in urban rivers using in situ cameras](https://mlsanity.com/paper/2510.23798/) (2025)
- [Which Leakage Types Matter? A Quantitative Landscape Across 2,047 Benchmark Datasets](https://mlsanity.com/paper/2604.04199/) (2026)
- [Data Leakage in Automotive Perception: Practitioners' Insights](https://mlsanity.com/paper/2604.06899/) (2026)
- [MedGate-Fusion: Integrating First-Encounter Semantic Narratives and Physiological Biomarkers for Prospective Stroke Risk Stratification](https://mlsanity.com/paper/2609.25272/) (2026)

## Abstract (arXiv)

Machine learning (ML) provides powerful tools for predictive modeling. ML's popularity stems from the promise of sample-level prediction with applications across a variety of fields from physics and marketing to healthcare. However, if not properly implemented and evaluated, ML pipelines may contain leakage typically resulting in overoptimistic performance estimates and failure to generalize to new data. This can have severe negative financial and societal implications. Our aim is to expand understanding associated with causes leading to leakage when designing, implementing, and evaluating ML pipelines. Illustrated by concrete examples, we provide a comprehensive overview and discussion of various types of leakage that may arise in ML pipelines.