# AlphaClean: Automatic Generation of Data Cleaning Pipelines

arXiv: https://arxiv.org/abs/1904.11827 · HTML: https://mlsanity.com/paper/1904.11827/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Apr 2019.

## Built on by (most cited first, 23 of 23)

- [Data-centric Artificial Intelligence: A Survey](https://mlsanity.com/paper/2303.10158/) (2023)
- [Training Data Influence Analysis and Estimation: A Survey](https://mlsanity.com/paper/2212.04612/) (2022)
- [How Much Automation Does a Data Scientist Want?](https://mlsanity.com/paper/2101.03970/) (2021)
- [README: Bridging Medical Jargon and Lay Understanding for Patient Education through Data-Centric NLP](https://mlsanity.com/paper/2312.15561/) (2023)
- [DC-Check: A Data-Centric AI checklist to guide the development of reliable machine learning systems](https://mlsanity.com/paper/2211.05764/) (2022)
- [Automated data processing and feature engineering for deep learning and big data applications: a survey](https://mlsanity.com/paper/2403.11395/) (2024)
- [Graph-based Confidence Calibration for Large Language Models](https://mlsanity.com/paper/2411.02454/) (2024)
- [DiffPrep: Differentiable Data Preprocessing Pipeline Search for Learning over Tabular Data](https://mlsanity.com/paper/2308.10915/) (2023)
- [Data Cleaning and Machine Learning: A Systematic Literature Review](https://mlsanity.com/paper/2310.01765/) (2023)
- [A Unified Framework for Task-Driven Data Quality Management](https://mlsanity.com/paper/2106.05484/) (2021)
- [Missing Values and Imputation in Healthcare Data: Can Interpretable Machine Learning Help?](https://mlsanity.com/paper/2304.11749/) (2023)
- [Auto-FP: An Experimental Study of Automated Feature Preprocessing for Tabular Data](https://mlsanity.com/paper/2310.02540/) (2023)
- [X Hacking: The Threat of Misguided AutoML](https://mlsanity.com/paper/2401.08513/) (2024)
- [Exploring the Intersection between Neural Architecture Search and Continual Learning](https://mlsanity.com/paper/2206.05625/) (2022)
- [Empowering Tabular Data Preparation with Language Models: Why and How?](https://mlsanity.com/paper/2508.01556/) (2025)
- [DataEvolver: Automatic Data Preparation for Large Language Models through Multi-Level Self-Evolving](https://mlsanity.com/paper/2606.07001/) (2026)
- [Adversarial Learning for Feature Shift Detection and Correction](https://mlsanity.com/paper/2312.04546/) (2023)
- [Data and System Perspectives of Sustainable Artificial Intelligence](https://mlsanity.com/paper/2501.07487/) (2025)
- [Man versus Machine: AutoML and Human Experts' Role in Phishing Detection](https://mlsanity.com/paper/2108.12193/) (2021)
- [From Strings to Data Science: a Practical Framework for Automated String Handling](https://mlsanity.com/paper/2111.01868/) (2021)
- [A Survey on Semantics in Automated Data Science](https://mlsanity.com/paper/2205.08018/) (2022)
- [Enhancing Sentiment Analysis Results through Outlier Detection Optimization](https://mlsanity.com/paper/2311.16185/) (2023)
- [Feature Shift Localization Network](https://mlsanity.com/paper/2506.09101/) (2025)

## Abstract (arXiv)

The analyst effort in data cleaning is gradually shifting away from the design of hand-written scripts to building and tuning complex pipelines of automated data cleaning libraries. Hyper-parameter tuning for data cleaning is very different than hyper-parameter tuning for machine learning since the pipeline components and objective functions have structure that tuning algorithms can exploit. This paper proposes a framework, called AlphaClean, that rethinks parameter tuning for data cleaning pipelines. AlphaClean provides users with a rich library to define data quality measures with weighted sums of SQL aggregate queries. AlphaClean applies generate-then-search framework where each pipelined cleaning operator contributes candidate transformations to a shared pool. Asynchronously, in separate threads, a search algorithm sequences them into cleaning pipelines that maximize the user-defined quality measures. This architecture allows AlphaClean to apply a number of optimizations including incremental evaluation of the quality measures and learning dynamic pruning rules to reduce the search space. Our experiments on real and synthetic benchmarks suggest that AlphaClean finds solutions of up-to 9x higher quality than naively applying state-of-the-art parameter tuning methods, is significantly more robust to straggling data cleaning methods and redundancy in the data cleaning library, and can incorporate state-of-the-art cleaning systems such as HoloClean as cleaning operators.