# REIN: A Comprehensive Benchmark Framework for Data Cleaning Methods in ML Pipelines

arXiv: https://arxiv.org/abs/2302.04702 · HTML: https://mlsanity.com/paper/2302.04702/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 56 references that are in the map.
- First posted on arXiv 9 Feb 2023.

## Builds on (most cited first, 13 of 13)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [Confident Learning: Estimating Uncertainty in Dataset Labels](https://mlsanity.com/paper/1911.00068/) (2019)
- [HoloClean: Holistic Data Repairs with Probabilistic Inference](https://mlsanity.com/paper/1702.00820/) (2017)
- [HoloDetect: Few-Shot Learning for Error Detection](https://mlsanity.com/paper/1904.02285/) (2019)
- [ZeroER: Entity Resolution using Zero Labeled Examples](https://mlsanity.com/paper/1908.06049/) (2019)
- [BoostClean: Automated Error Detection and Repair for Machine Learning](https://mlsanity.com/paper/1711.01299/) (2017)
- [Nearest Neighbor Classifiers over Incomplete Information: From Certain Answers to Certain Predictions](https://mlsanity.com/paper/2005.05117/) (2020)
- [Picket: Guarding Against Corrupted Data in Tabular Data during Learning and Inference](https://mlsanity.com/paper/2006.04730/) (2020)
- [CleanML: A Study for Evaluating the Impact of Data Cleaning on ML Classification Tasks](https://mlsanity.com/paper/1904.09483/) (2019)
- [A Survey on Data Cleaning Methods for Improved Machine Learning Model Performance](https://mlsanity.com/paper/2109.07127/) (2021)
- [Learning Over Dirty Data Without Cleaning](https://mlsanity.com/paper/2004.02308/) (2020)

## Built on by (most cited first, 10 of 10)

- [Data-centric Artificial Intelligence: A Survey](https://mlsanity.com/paper/2303.10158/) (2023)
- [AnyMatch -- Efficient Zero-Shot Entity Matching with a Small Language Model](https://mlsanity.com/paper/2409.04073/) (2024)
- [Open-Source Drift Detection Tools in Action: Insights from Two Use Cases](https://mlsanity.com/paper/2404.18673/) (2024)
- [CleanPatrick: A Benchmark for Image Data Cleaning](https://mlsanity.com/paper/2505.11034/) (2025)
- [Still More Shades of Null: An Evaluation Suite for Responsible Missing Value Imputation](https://mlsanity.com/paper/2409.07510/) (2024)
- [Stress-Testing ML Pipelines with Adversarial Data Corruption](https://mlsanity.com/paper/2506.01230/) (2025)
- [Quality Assessment of Tabular Data using Large Language Models and Code Generation](https://mlsanity.com/paper/2509.10572/) (2025)
- [AutoCure: Automated Tabular Data Curation Technique for ML Pipelines](https://mlsanity.com/paper/2304.13636/) (2023)
- [A Comparative Analysis of Influence Signals for Data Debugging](https://mlsanity.com/paper/2506.11584/) (2025)
- [Quality-Driven Selective Mutation for Deep Learning](https://mlsanity.com/paper/2604.22640/) (2026)

## Abstract (arXiv)

Nowadays, machine learning (ML) plays a vital role in many aspects of our daily life. In essence, building well-performing ML applications requires the provision of high-quality data throughout the entire life-cycle of such applications. Nevertheless, most of the real-world tabular data suffer from different types of discrepancies, such as missing values, outliers, duplicates, pattern violation, and inconsistencies. Such discrepancies typically emerge while collecting, transferring, storing, and/or integrating the data. To deal with these discrepancies, numerous data cleaning methods have been introduced. However, the majority of such methods broadly overlook the requirements imposed by downstream ML models. As a result, the potential of utilizing these data cleaning methods in ML pipelines is predominantly unrevealed. In this work, we introduce a comprehensive benchmark, called REIN1, to thoroughly investigate the impact of data cleaning methods on various ML models. Through the benchmark, we provide answers to important research questions, e.g., where and whether data cleaning is a necessary step in ML pipelines. To this end, the benchmark examines 38 simple and advanced error detection and repair methods. To evaluate these methods, we utilized a wide collection of ML models trained on 14 publicly-available datasets covering different domains and encompassing realistic as well as synthetic error profiles.