# LeakageDetector: An Open Source Data Leakage Analysis Tool in Machine Learning Pipelines

arXiv: https://arxiv.org/abs/2503.14723 · HTML: https://mlsanity.com/paper/2503.14723/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Mar 2025.

## Built on by (most cited first, 2 of 2)

- [A Grammar of Machine Learning Workflows: Rejecting Data Leakage at Call Time](https://mlsanity.com/paper/2603.10742/) (2026)
- [Improving AGI Evaluation: A Data Science Perspective](https://mlsanity.com/paper/2510.01687/) (2025)

## Abstract (arXiv)

Code quality is of paramount importance in all types of software development settings. Our work seeks to enable Machine Learning (ML) engineers to write better code by helping them find and fix instances of Data Leakage in their models. Data Leakage often results from bad practices in writing ML code. As a result, the model effectively ''memorizes'' the data on which it trains, leading to an overly optimistic estimate of the model performance and an inability to make generalized predictions. ML developers must carefully separate their data into training, evaluation, and test sets to avoid introducing Data Leakage into their code. Training data should be used to train the model, evaluation data should be used to repeatedly confirm a model's accuracy, and test data should be used only once to determine the accuracy of a production-ready model. In this paper, we develop LEAKAGEDETECTOR, a Python plugin for the PyCharm IDE that identifies instances of Data Leakage in ML code and provides suggestions on how to remove the leakage.