# Identifying Mislabeled Instances in Classification Datasets

arXiv: https://arxiv.org/abs/1912.05283 · HTML: https://mlsanity.com/paper/1912.05283/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2019.

## Built on by (most cited first, 26 of 26)

- [Evaluations of Machine Learning Privacy Defenses are Misleading](https://mlsanity.com/paper/2404.17399/) (2024)
- [Why is SAM Robust to Label Noise?](https://mlsanity.com/paper/2405.03676/) (2024)
- [Detecting Label Errors by using Pre-Trained Language Models](https://mlsanity.com/paper/2205.12702/) (2022)
- [Improving Robustness with Adaptive Weight Decay](https://mlsanity.com/paper/2210.00094/) (2022)
- [The METRIC-framework for assessing data quality for trustworthy AI in medicine: a systematic review](https://mlsanity.com/paper/2402.13635/) (2024)
- [Detecting Label Errors in Token Classification Data](https://mlsanity.com/paper/2210.03920/) (2022)
- [Identifying Incorrect Annotations in Multi-Label Classification Data](https://mlsanity.com/paper/2211.13895/) (2022)
- [Estimating label quality and errors in semantic segmentation data via any model](https://mlsanity.com/paper/2307.05080/) (2023)
- [ObjectLab: Automated Diagnosis of Mislabeled Images in Object Detection Data](https://mlsanity.com/paper/2309.00832/) (2023)
- [Outlier Gradient Analysis: Efficiently Identifying Detrimental Training Samples for Deep Learning Models](https://mlsanity.com/paper/2405.03869/) (2024)
- [Efficient Adversarial Training With Data Pruning](https://mlsanity.com/paper/2207.00694/) (2022)
- [Active Label Correction for Semantic Segmentation with Foundation Models](https://mlsanity.com/paper/2403.10820/) (2024)
- [Learning Sample Reweighting for Accuracy and Adversarial Robustness](https://mlsanity.com/paper/2210.11513/) (2022)
- [Assessing the Quality of the Datasets by Identifying Mislabeled Samples](https://mlsanity.com/paper/2109.05000/) (2021)
- [Gradient-Based Automated Iterative Recovery for Parameter-Efficient Tuning](https://mlsanity.com/paper/2302.06598/) (2023)
- [Towards understanding neural collapse in supervised contrastive learning with the information bottleneck method](https://mlsanity.com/paper/2305.11957/) (2023)
- [On Evaluation of Document Classification using RVL-CDIP](https://mlsanity.com/paper/2306.12550/) (2023)
- [Training Dynamic based data filtering may not work for NLP datasets](https://mlsanity.com/paper/2109.09191/) (2021)
- [GenColor: Generative Color-Concept Association in Visual Design](https://mlsanity.com/paper/2503.03236/) (2025)
- [Identify ambiguous tasks combining crowdsourced labels by weighting Areas Under the Margin](https://mlsanity.com/paper/2209.15380/) (2022)
- [Defending Against Adversarial Denial-of-Service Data Poisoning Attacks](https://mlsanity.com/paper/2104.06744/) (2021)
- [Detecting Errors in a Numerical Response via any Regression Model](https://mlsanity.com/paper/2305.16583/) (2023)
- [Learning to Detect Label Errors by Making Them: A Method for Segmentation and Object Detection Datasets](https://mlsanity.com/paper/2508.17930/) (2025)
- [An Assessment of Human vs. Model Uncertainty in Soft-Label Learning and Calibration](https://mlsanity.com/paper/2605.18648/) (2026)
- [Analyzing Training-Free Corruption Detection for Object Detection Datasets](https://mlsanity.com/paper/2606.10666/) (2026)
- [Revising RVL-CDIP: Quantifying Errors and Test-Train Overlap](https://mlsanity.com/paper/2606.31446/) (2026)

## Abstract (arXiv)

A key requirement for supervised machine learning is labeled training data, which is created by annotating unlabeled data with the appropriate class. Because this process can in many cases not be done by machines, labeling needs to be performed by human domain experts. This process tends to be expensive both in time and money, and is prone to errors. Additionally, reviewing an entire labeled dataset manually is often prohibitively costly, so many real world datasets contain mislabeled instances. To address this issue, we present in this paper a non-parametric end-to-end pipeline to find mislabeled instances in numerical, image and natural language datasets. We evaluate our system quantitatively by adding a small number of label noise to 29 datasets, and show that we find mislabeled instances with an average precision of more than 0.84 when reviewing our system's top 1% recommendation. We then apply our system to publicly available datasets and find mislabeled instances in CIFAR-100, Fashion-MNIST, and others. Finally, we publish the code and an applicable implementation of our approach.