# Detecting Corrupted Labels Without Training a Model to Predict

arXiv: https://arxiv.org/abs/2110.06283 · HTML: https://mlsanity.com/paper/2110.06283/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 104 references that are in the map.
- First posted on arXiv 12 Oct 2021.

## Builds on (most cited first, 30 of 53)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Estimating Training Data Influence by Tracing Gradient Descent](https://mlsanity.com/paper/2002.08484/) (2020)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [Just Train Twice: Improving Group Robustness without Training Group Information](https://mlsanity.com/paper/2107.09044/) (2021)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [Early-Learning Regularization Prevents Memorization of Noisy Labels](https://mlsanity.com/paper/2007.00151/) (2020)
- [Confident Learning: Estimating Uncertainty in Dataset Labels](https://mlsanity.com/paper/1911.00068/) (2019)
- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [How does Disagreement Help Generalization against Label Corruption?](https://mlsanity.com/paper/1901.04215/) (2019)
- [Combating noisy labels by agreement: A joint training method with co-regularization](https://mlsanity.com/paper/2003.02752/) (2020)
- [Are Anchor Points Really Indispensable in Label-Noise Learning?](https://mlsanity.com/paper/1906.00189/) (2019)
- [To Trust Or Not To Trust A Classifier](https://mlsanity.com/paper/1805.11783/) (2018)
- [TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech](https://mlsanity.com/paper/2007.06028/) (2020)
- [Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks](https://mlsanity.com/paper/1903.11680/) (2019)
- [Learning with Instance-Dependent Label Noise: A Sample Sieve Approach](https://mlsanity.com/paper/2010.02347/) (2020)
- [Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates](https://mlsanity.com/paper/1910.03231/) (2019)
- [A Survey of Label-noise Representation Learning: Past, Present and Future](https://mlsanity.com/paper/2011.04406/) (2020)
- [Understanding and Improving Early Stopping for Learning with Noisy Labels](https://mlsanity.com/paper/2106.15853/) (2021)

## Built on by (most cited first, 30 of 58)

- [Mitigating Neural Network Overconfidence with Logit Normalization](https://mlsanity.com/paper/2205.09310/) (2022)
- [LLM Unlearning via Loss Adjustment with Only Forget Data](https://mlsanity.com/paper/2410.11143/) (2024)
- [Mitigating Memorization of Noisy Labels by Clipping the Model Prediction](https://mlsanity.com/paper/2212.04055/) (2022)
- [Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning](https://mlsanity.com/paper/2502.01968/) (2025)
- [Measuring and Reducing LLM Hallucination without Gold-Standard Answers](https://mlsanity.com/paper/2402.10412/) (2024)
- [To Aggregate or Not? Learning with Separate Noisy Labels](https://mlsanity.com/paper/2206.07181/) (2022)
- [Beyond Images: Label Noise Transition Matrix Estimation for Tasks with Lower-Quality Features](https://mlsanity.com/paper/2202.01273/) (2022)
- [Understanding Instance-Level Impact of Fairness Constraints](https://mlsanity.com/paper/2206.15437/) (2022)
- [Unmasking and Improving Data Credibility: A Study with Datasets for Training Harmless Language Models](https://mlsanity.com/paper/2311.11202/) (2023)
- [Open-Sampling: Exploring Out-of-Distribution data for Re-balancing Long-tailed datasets](https://mlsanity.com/paper/2206.08802/) (2022)
- [Automatic Dataset Construction (ADC): Sample Collection, Data Curation, and Beyond](https://mlsanity.com/paper/2408.11338/) (2024)
- [Weak Proxies are Sufficient and Preferable for Fairness with Missing Sensitive Attributes](https://mlsanity.com/paper/2210.03175/) (2022)
- [AQuA: A Benchmarking Tool for Label Quality Assessment](https://mlsanity.com/paper/2306.09467/) (2023)
- [On the Noise Robustness of In-Context Learning for Text Generation](https://mlsanity.com/paper/2405.17264/) (2024)
- [CLIPCleaner: Cleaning Noisy Labels with CLIP](https://mlsanity.com/paper/2408.10012/) (2024)
- [On the Importance of Feature Separability in Predicting Out-Of-Distribution Error](https://mlsanity.com/paper/2303.15488/) (2023)
- [Neighborhood-Regularized Self-Training for Learning with Few Labels](https://mlsanity.com/paper/2301.03726/) (2023)
- [Labeling Chaos to Learning Harmony: Federated Learning with Noisy Labels](https://mlsanity.com/paper/2208.09378/) (2022)
- [AFreeCA: Annotation-Free Counting for All](https://mlsanity.com/paper/2403.04943/) (2024)
- [VDC: Versatile Data Cleanser based on Visual-Linguistic Inconsistency by Multimodal Large Language Models](https://mlsanity.com/paper/2309.16211/) (2023)
- [Rethinking model prototyping through the MedMNIST+ dataset collection](https://mlsanity.com/paper/2404.15786/) (2024)
- [Learning Discriminative Dynamics with Label Corruption for Noisy Label Detection](https://mlsanity.com/paper/2405.19902/) (2024)
- [The Importance of Human-Labeled Data in the Era of LLMs](https://mlsanity.com/paper/2306.14910/) (2023)
- [Fine tuning Pre trained Models for Robustness Under Noisy Labels](https://mlsanity.com/paper/2310.17668/) (2023)
- [GraphCleaner: Detecting Mislabelled Samples in Popular Graph Learning Benchmarks](https://mlsanity.com/paper/2306.00015/) (2023)
- [Combating Label Noise With A General Surrogate Model For Sample Selection](https://mlsanity.com/paper/2310.10463/) (2023)
- [LEMoN: Label Error Detection using Multimodal Neighbors](https://mlsanity.com/paper/2407.18941/) (2024)
- [Bounded and Uniform Energy-based Out-of-distribution Detection for Graphs](https://mlsanity.com/paper/2504.13429/) (2025)
- [Fairness Improves Learning from Noisily Labeled Long-Tailed Data](https://mlsanity.com/paper/2303.12291/) (2023)
- [Inconsistency-Based Data-Centric Active Open-Set Annotation](https://mlsanity.com/paper/2401.04923/) (2024)

## Abstract (arXiv)

Label noise in real-world datasets encodes wrong correlation patterns and impairs the generalization of deep neural networks (DNNs). It is critical to find efficient ways to detect corrupted patterns. Current methods primarily focus on designing robust training techniques to prevent DNNs from memorizing corrupted patterns. These approaches often require customized training processes and may overfit corrupted patterns, leading to a performance drop in detection. In this paper, from a more data-centric perspective, we propose a training-free solution to detect corrupted labels. Intuitively, ``closer'' instances are more likely to share the same clean label. Based on the neighborhood information, we propose two methods: the first one uses ``local voting" via checking the noisy label consensuses of nearby features. The second one is a ranking-based approach that scores each instance and filters out a guaranteed number of instances that are likely to be corrupted. We theoretically analyze how the quality of features affects the local voting and provide guidelines for tuning neighborhood size. We also prove the worst-case error bound for the ranking-based method. Experiments with both synthetic and real-world label noise demonstrate our training-free solutions consistently and significantly improve most of the training-based baselines. Code is available at github.com/UCSC-REAL/SimiFeat.