# Data Quality Toolkit: Automatic assessment of data quality and remediation for machine learning datasets

arXiv: https://arxiv.org/abs/2108.05935 · HTML: https://mlsanity.com/paper/2108.05935/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 22 references that are in the map.
- First posted on arXiv 12 Aug 2021.

## Builds on (most cited first, 5 of 5)

- [Confident Learning: Estimating Uncertainty in Dataset Labels](https://mlsanity.com/paper/1911.00068/) (2019)
- [AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias](https://mlsanity.com/paper/1810.01943/) (2018)
- [An ADMM Based Framework for AutoML Pipeline Configuration](https://mlsanity.com/paper/1905.00424/) (2019)
- [Data Readiness Report](https://mlsanity.com/paper/2010.07213/) (2020)
- [Comparison of Outlier Detection Techniques for Structured Data](https://mlsanity.com/paper/2106.08779/) (2021)

## Built on by (most cited first, 18 of 18)

- [Reimagining Synthetic Tabular Data Generation through Data-Centric AI: A Comprehensive Benchmark](https://mlsanity.com/paper/2310.16981/) (2023)
- [Data-IQ: Characterizing subgroups with heterogeneous outcomes in tabular data](https://mlsanity.com/paper/2210.13043/) (2022)
- [Data Readiness for AI: A 360-Degree Survey](https://mlsanity.com/paper/2404.05779/) (2024)
- [Dissecting Sample Hardness: A Fine-Grained Analysis of Hardness Characterization Methods for Data-Centric AI](https://mlsanity.com/paper/2403.04551/) (2024)
- [AI Data Readiness Inspector (AIDRIN) for Quantitative Assessment of Data Readiness for AI](https://mlsanity.com/paper/2406.19256/) (2024)
- [Matchmaker: Self-Improving Large Language Model Programs for Schema Matching](https://mlsanity.com/paper/2410.24105/) (2024)
- [TRIAGE: Characterizing and auditing training data for improved regression](https://mlsanity.com/paper/2310.18970/) (2023)
- [DCA-Bench: A Benchmark for Dataset Curation Agents](https://mlsanity.com/paper/2406.07275/) (2024)
- [Data Smells: Categories, Causes and Consequences, and Detection of Suspicious Data in AI-based Systems](https://mlsanity.com/paper/2203.10384/) (2022)
- [MLOps Spanning Whole Machine Learning Life Cycle: A Survey](https://mlsanity.com/paper/2304.07296/) (2023)
- [You can't handle the (dirty) truth: Data-centric insights improve pseudo-labeling](https://mlsanity.com/paper/2406.13733/) (2024)
- [Model-Based Data-Centric AI: Bridging the Divide Between Academic Ideals and Industrial Pragmatism](https://mlsanity.com/paper/2403.01832/) (2024)
- [Analytics Modelling over Multiple Datasets using Vector Embeddings](https://mlsanity.com/paper/2502.17060/) (2025)
- [Data Quality Awareness: A Journey from Traditional Data Management to Data Science Systems](https://mlsanity.com/paper/2411.03007/) (2024)
- [Assessing the Impact of the Quality of Textual Data on Feature Representation and Machine Learning Models](https://mlsanity.com/paper/2502.08669/) (2025)
- [CADRE: Customizable Assurance of Data Readiness in Privacy-Preserving Federated Learning](https://mlsanity.com/paper/2505.23849/) (2025)
- [d-DQIVAR: Data-centric Visual Analytics and Reasoning for Data Quality Improvement](https://mlsanity.com/paper/2507.11960/) (2025)
- [SciHorizon-DataEVA: An Agentic System for AI-Readiness Evaluation of Heterogeneous Scientific Data](https://mlsanity.com/paper/2604.26645/) (2026)

## Abstract (arXiv)

The quality of training data has a huge impact on the efficiency, accuracy and complexity of machine learning tasks. Various tools and techniques are available that assess data quality with respect to general cleaning and profiling checks. However these techniques are not applicable to detect data issues in the context of machine learning tasks, like noisy labels, existence of overlapping classes etc. We attempt to re-look at the data quality issues in the context of building a machine learning pipeline and build a tool that can detect, explain and remediate issues in the data, and systematically and automatically capture all the changes applied to the data. We introduce the Data Quality Toolkit for machine learning as a library of some key quality metrics and relevant remediation techniques to analyze and enhance the readiness of structured training datasets for machine learning projects. The toolkit can reduce the turn-around times of data preparation pipelines and streamline the data quality assessment process. Our toolkit is publicly available via IBM API Hub [1] platform, any developer can assess the data quality using the IBM's Data Quality for AI apis [2]. Detailed tutorials are also available on IBM Learning Path [3].