# DiffPrep: Differentiable Data Preprocessing Pipeline Search for Learning over Tabular Data

arXiv: https://arxiv.org/abs/2308.10915 · HTML: https://mlsanity.com/paper/2308.10915/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 48 references that are in the map.
- First posted on arXiv 20 Aug 2023.

## Builds on (most cited first, 13 of 13)

- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [AutoML: A Survey of the State-of-the-Art](https://mlsanity.com/paper/1908.00709/) (2019)
- [Forward and Reverse Gradient-Based Hyperparameter Optimization](https://mlsanity.com/paper/1703.01785/) (2017)
- [Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learning](https://mlsanity.com/paper/2007.04074/) (2020)
- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [DeepPermNet: Visual Permutation Learning](https://mlsanity.com/paper/1704.02729/) (2017)
- [BoostClean: Automated Error Detection and Repair for Machine Learning](https://mlsanity.com/paper/1711.01299/) (2017)
- [AlphaClean: Automatic Generation of Data Cleaning Pipelines](https://mlsanity.com/paper/1904.11827/) (2019)
- [Evolving Search Space for Neural Architecture Search](https://mlsanity.com/paper/2011.10904/) (2020)
- [Nearest Neighbor Classifiers over Incomplete Information: From Certain Answers to Certain Predictions](https://mlsanity.com/paper/2005.05117/) (2020)
- [DiffML: End-to-end Differentiable ML Pipelines](https://mlsanity.com/paper/2207.01269/) (2022)
- [Two-stage Optimization for Machine Learning Workflow](https://mlsanity.com/paper/1907.00678/) (2019)

## Built on by (most cited first, 9 of 9)

- [Automated data processing and feature engineering for deep learning and big data applications: a survey](https://mlsanity.com/paper/2403.11395/) (2024)
- [Stress-Testing ML Pipelines with Adversarial Data Corruption](https://mlsanity.com/paper/2506.01230/) (2025)
- [LLaPipe: LLM-Guided Reinforcement Learning for Automated Data Preparation Pipeline Construction](https://mlsanity.com/paper/2507.13712/) (2025)
- [LLM-AutoDP: Automatic Data Processing via LLM Agents for Model Fine-tuning](https://mlsanity.com/paper/2601.20375/) (2026)
- [Can AI Evaluate AI Scientists? A Benchmarking Study of Autonomous Research Generation Systems Using Automated Multi-Model Review](https://mlsanity.com/paper/2607.28631/) (2026)
- [SoftPipe: A Soft-Guided Reinforcement Learning Framework for Automated Data Preparation](https://mlsanity.com/paper/2507.13710/) (2025)
- [Model-Aware Data Cleaning for Tabular Foundation Models](https://mlsanity.com/paper/2604.25154/) (2026)
- [FlowPipe: LLM-Enhanced Conditional Generative Flow Networks for Data Preparation Pipeline Construction](https://mlsanity.com/paper/2606.24679/) (2026)
- [ML Engineering Agents - A Data Management Perspective](https://mlsanity.com/paper/2610.16919/) (2026)

## Abstract (arXiv)

Data preprocessing is a crucial step in the machine learning process that transforms raw data into a more usable format for downstream ML models. However, it can be costly and time-consuming, often requiring the expertise of domain experts. Existing automated machine learning (AutoML) frameworks claim to automate data preprocessing. However, they often use a restricted search space of data preprocessing pipelines which limits the potential performance gains, and they are often too slow as they require training the ML model multiple times. In this paper, we propose DiffPrep, a method that can automatically and efficiently search for a data preprocessing pipeline for a given tabular dataset and a differentiable ML model such that the performance of the ML model is maximized. We formalize the problem of data preprocessing pipeline search as a bi-level optimization problem. To solve this problem efficiently, we transform and relax the discrete, non-differential search space into a continuous and differentiable one, which allows us to perform the pipeline search using gradient descent with training the ML model only once. Our experiments show that DiffPrep achieves the best test accuracy on 15 out of the 18 real-world datasets evaluated and improves the model's test accuracy by up to 6.6 percentage points.