# DiffML: End-to-end Differentiable ML Pipelines

arXiv: https://arxiv.org/abs/2207.01269 · HTML: https://mlsanity.com/paper/2207.01269/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 39 references that are in the map.
- First posted on arXiv 4 Jul 2022.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [GAIN: Missing Data Imputation using Generative Adversarial Nets](https://mlsanity.com/paper/1806.02920/) (2018)
- [Deep Learning without Poor Local Minima](https://mlsanity.com/paper/1605.07110/) (2016)
- [HoloClean: Holistic Data Repairs with Probabilistic Inference](https://mlsanity.com/paper/1702.00820/) (2017)
- [Improving Deep Learning using Generic Data Augmentation](https://mlsanity.com/paper/1708.06020/) (2017)
- [Depth Creates No Bad Local Minima](https://mlsanity.com/paper/1702.08580/) (2017)
- [ARM-Net: Adaptive Relation Modeling Network for Structured Data](https://mlsanity.com/paper/2107.01830/) (2021)
- [ARDA: Automatic Relational Data Augmentation for Machine Learning](https://mlsanity.com/paper/2003.09758/) (2020)
- [Sample Debiasing in the Themis Open World Database System (Extended Version)](https://mlsanity.com/paper/2002.09799/) (2020)
- [ReStore -- Neural Data Completion for Relational Databases](https://mlsanity.com/paper/2105.12457/) (2021)

## Built on by (most cited first, 7 of 7)

- [Automated data processing and feature engineering for deep learning and big data applications: a survey](https://mlsanity.com/paper/2403.11395/) (2024)
- [DiffPrep: Differentiable Data Preprocessing Pipeline Search for Learning over Tabular Data](https://mlsanity.com/paper/2308.10915/) (2023)
- [AutoML in Heavily Constrained Applications](https://mlsanity.com/paper/2306.16913/) (2023)
- [LLaPipe: LLM-Guided Reinforcement Learning for Automated Data Preparation Pipeline Construction](https://mlsanity.com/paper/2507.13712/) (2025)
- [Exploring End-to-end Differentiable Neural Charged Particle Tracking -- A Loss Landscape Perspective](https://mlsanity.com/paper/2407.13420/) (2024)
- [FlowPipe: LLM-Enhanced Conditional Generative Flow Networks for Data Preparation Pipeline Construction](https://mlsanity.com/paper/2606.24679/) (2026)
- [ML Engineering Agents - A Data Management Perspective](https://mlsanity.com/paper/2610.16919/) (2026)

## Abstract (arXiv)

In this paper, we present our vision of differentiable ML pipelines called DiffML to automate the construction of ML pipelines in an end-to-end fashion. The idea is that DiffML allows to jointly train not just the ML model itself but also the entire pipeline including data preprocessing steps, e.g., data cleaning, feature selection, etc. Our core idea is to formulate all pipeline steps in a differentiable way such that the entire pipeline can be trained using backpropagation. However, this is a non-trivial problem and opens up many new research questions. To show the feasibility of this direction, we demonstrate initial ideas and a general principle of how typical preprocessing steps such as data cleaning, feature selection and dataset selection can be formulated as differentiable programs and jointly learned with the ML model. Moreover, we discuss a research roadmap and core challenges that have to be systematically tackled to enable fully differentiable ML pipelines.