# Detecting Label Noise via Leave-One-Out Cross-Validation

arXiv: https://arxiv.org/abs/2103.11352 · HTML: https://mlsanity.com/paper/2103.11352/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 43 references that are in the map.
- First posted on arXiv 21 Mar 2021.

## Builds on (most cited first, 15 of 15)

- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [How does Disagreement Help Generalization against Label Corruption?](https://mlsanity.com/paper/1901.04215/) (2019)
- [Joint Optimization Framework for Learning with Noisy Labels](https://mlsanity.com/paper/1803.11364/) (2018)
- [Combating noisy labels by agreement: A joint training method with co-regularization](https://mlsanity.com/paper/2003.02752/) (2020)
- [Normalized Loss Functions for Deep Learning with Noisy Labels](https://mlsanity.com/paper/2006.13554/) (2020)
- [Deep learning with noisy labels: exploring techniques and remedies in medical image analysis](https://mlsanity.com/paper/1912.02911/) (2019)
- [Learning from Noisy Labels with Distillation](https://mlsanity.com/paper/1703.02391/) (2017)
- [GPflow: A Gaussian process library using TensorFlow](https://mlsanity.com/paper/1610.08733/) (2016)
- [Digital Twin: Values, Challenges and Enablers](https://mlsanity.com/paper/1910.01719/) (2019)
- [A Topological Filter for Learning with Label Noise](https://mlsanity.com/paper/2012.04835/) (2020)
- [Chained Gaussian Processes](https://mlsanity.com/paper/1604.05263/) (2016)
- [Prediction of Atomization Energy Using Graph Kernel and Active Learning](https://mlsanity.com/paper/1810.07310/) (2018)
- [A Study of Clustering Techniques and Hierarchical Matrix Formats for Kernel Ridge Regression](https://mlsanity.com/paper/1803.10274/) (2018)

## Built on by (most cited first, 2 of 2)

- [Slight Corruption in Pre-training Data Makes Better Diffusion Models](https://mlsanity.com/paper/2405.20494/) (2024)
- [FPR Estimation for Fraud Detection in the Presence of Class-Conditional Label Noise](https://mlsanity.com/paper/2308.02695/) (2023)

## Abstract (arXiv)

We present a simple algorithm for identifying and correcting real-valued noisy labels from a mixture of clean and corrupted sample points using Gaussian process regression. A heteroscedastic noise model is employed, in which additive Gaussian noise terms with independent variances are associated with each and all of the observed labels. Optimizing the noise model using maximum likelihood estimation leads to the containment of the GPR model's predictive error by the posterior standard deviation in leave-one-out cross-validation. A multiplicative update scheme is proposed for solving the maximum likelihood estimation problem under non-negative constraints. While we provide proof of convergence for certain special cases, the multiplicative scheme has empirically demonstrated monotonic convergence behavior in virtually all our numerical experiments. We show that the presented method can pinpoint corrupted sample points and lead to better regression models when trained on synthetic and real-world scientific data sets.