# Detecting and Rectifying Noisy Labels: A Similarity-based Approach

arXiv: https://arxiv.org/abs/2509.23964 · HTML: https://mlsanity.com/paper/2509.23964/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 40 references that are in the map.
- First posted on arXiv 28 Sep 2025.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [Estimating Training Data Influence by Tracing Gradient Descent](https://mlsanity.com/paper/2002.08484/) (2020)
- [Confident Learning: Estimating Uncertainty in Dataset Labels](https://mlsanity.com/paper/1911.00068/) (2019)
- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [Are we done with ImageNet?](https://mlsanity.com/paper/2006.07159/) (2020)
- [Influence Functions in Deep Learning Are Fragile](https://mlsanity.com/paper/2006.14651/) (2020)
- [Deep Learning is Robust to Massive Label Noise](https://mlsanity.com/paper/1705.10694/) (2017)
- [Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks](https://mlsanity.com/paper/2010.02066/) (2020)
- [Input Similarity from the Neural Network Perspective](https://mlsanity.com/paper/2102.05262/) (2021)
- [A Topological Filter for Learning with Label Noise](https://mlsanity.com/paper/2012.04835/) (2020)
- [Deep k-NN for Noisy Labels](https://mlsanity.com/paper/2004.12289/) (2020)
- [An Empirical Comparison of Instance Attribution Methods for NLP](https://mlsanity.com/paper/2104.04128/) (2021)
- [Detecting Label Errors in Token Classification Data](https://mlsanity.com/paper/2210.03920/) (2022)
- [Identifying Incorrect Annotations in Multi-Label Classification Data](https://mlsanity.com/paper/2211.13895/) (2022)
- [Fast Rates for a kNN Classifier Robust to Unknown Asymmetric Label Noise](https://mlsanity.com/paper/1906.04542/) (2019)
- [Towards Using Data-Influence Methods to Detect Noisy Samples in Source Code Corpora](https://mlsanity.com/paper/2205.13022/) (2022)
- [Class based Influence Functions for Error Detection](https://mlsanity.com/paper/2305.01384/) (2023)

## Abstract (arXiv)

Label noise in datasets could significantly damage the performance and robustness of deep neural networks (DNNs) trained on these datasets. As the size of modern DNNs grows, there is a growing demand for automated tools for detecting such errors. In this paper, we propose post-hoc, model-agnostic noise detection and rectification methods utilizing the penultimate feature from a DNN. Our idea is based on the observation that the similarity between the penultimate feature of a mislabeled data point and its true class data points is higher than that for data points from other classes, making the probability of label occurrence within a tight, similar cluster informative for detecting and rectifying errors. Through theoretical and empirical analyses, we demonstrate that our approach achieves high detection performance across diverse, realistic noise scenarios and can automatically rectify these errors to improve dataset quality. Our implementation is available at https://anonymous.4open.science/r/noise-detection-and-rectification-AD8E.