# A Grammar of Machine Learning Workflows: Rejecting Data Leakage at Call Time

arXiv: https://arxiv.org/abs/2603.10742 · HTML: https://mlsanity.com/paper/2603.10742/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 53 references that are in the map.
- First posted on arXiv 11 Mar 2026.

## Builds on (most cited first, 8 of 8)

- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [The Ladder: A Reliable Leaderboard for Machine Learning Competitions](https://mlsanity.com/paper/1502.04585/) (2015)
- [Cross-validation: what does it estimate and how well does it do it?](https://mlsanity.com/paper/2104.00673/) (2021)
- [Leakage and the Reproducibility Crisis in ML-based Science](https://mlsanity.com/paper/2207.07048/) (2022)
- [The Machine Learning Bazaar: Harnessing the ML Ecosystem for Effective System Development](https://mlsanity.com/paper/1905.08942/) (2019)
- [Data Leakage in Notebooks: Static Detection and Better Processes](https://mlsanity.com/paper/2209.03345/) (2022)
- [LeakageDetector: An Open Source Data Leakage Analysis Tool in Machine Learning Pipelines](https://mlsanity.com/paper/2503.14723/) (2025)
- [LeakageDetector 2.0: Analyzing Data Leakage in Jupyter-Driven Machine Learning Pipelines](https://mlsanity.com/paper/2509.15971/) (2025)

## Built on by (most cited first, 1 of 1)

- [Which Leakage Types Matter? A Quantitative Landscape Across 2,047 Benchmark Datasets](https://mlsanity.com/paper/2604.04199/) (2026)

## Abstract (arXiv)

Data leakage has been identified in 648 published papers across 30 scientific fields. The knowledge to prevent it has existed for over a decade; the problem persists because the tools do not enforce what the textbooks teach. This paper presents a grammar (eight typed primitives connected by a directed acyclic graph with four hard constraints) that makes the most damaging leakage types structurally unrepresentable within the grammar's scope. The core mechanism is a terminal assessment gate: the first call-time-enforced evaluate/assess boundary documented in the peer-reviewed ML methodology literature (to my knowledge, as of May 2026), backed by a specification precise enough for independent reimplementation. A companion landscape study across 2,047 datasets grounds the constraints in measured effect sizes. Two reference implementations (Python, R) are available.