# Learning with Confident Examples: Rank Pruning for Robust Classification with Noisy Labels

arXiv: https://arxiv.org/abs/1705.01936 · HTML: https://mlsanity.com/paper/1705.01936/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 May 2017.

## Built on by (most cited first, 30 of 52)

- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [Provably End-to-end Label-Noise Learning without Anchor Points](https://mlsanity.com/paper/2102.02400/) (2021)
- [DataPerf: Benchmarks for Data-Centric AI Development](https://mlsanity.com/paper/2207.10062/) (2022)
- [UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learning](https://mlsanity.com/paper/2203.14542/) (2022)
- [Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labels](https://mlsanity.com/paper/2105.04522/) (2021)
- [Clusterability as an Alternative to Anchor Points When Learning with Noisy Labels](https://mlsanity.com/paper/2102.05291/) (2021)
- [Instance-Dependent Label-Noise Learning with Manifold-Regularized Transition Matrix Estimation](https://mlsanity.com/paper/2206.02791/) (2022)
- [Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning](https://mlsanity.com/paper/2502.01968/) (2025)
- [Dist-PU: Positive-Unlabeled Learning from a Label Distribution Perspective](https://mlsanity.com/paper/2212.02801/) (2022)
- [Joint Negative and Positive Learning for Noisy Labels](https://mlsanity.com/paper/2104.06574/) (2021)
- [Identifiability of Label Noise Transition Matrix](https://mlsanity.com/paper/2202.02016/) (2022)
- [Estimating Instance-dependent Bayes-label Transition Matrix using a Deep Neural Network](https://mlsanity.com/paper/2105.13001/) (2021)
- [Consistency Regularization Can Improve Robustness to Label Noise](https://mlsanity.com/paper/2110.01242/) (2021)
- [On the Effects of Different Types of Label Noise in Multi-Label Remote Sensing Image Classification](https://mlsanity.com/paper/2207.13975/) (2022)
- [CTRL: Clustering Training Losses for Label Error Detection](https://mlsanity.com/paper/2208.08464/) (2022)
- [AQuA: A Benchmarking Tool for Label Quality Assessment](https://mlsanity.com/paper/2306.09467/) (2023)
- [A Short Survey on Importance Weighting for Machine Learning](https://mlsanity.com/paper/2403.10175/) (2024)
- [Label Cleaning Multiple Instance Learning: Refining Coarse Annotations on Single Whole-Slide Images](https://mlsanity.com/paper/2109.10778/) (2021)
- [Breaking the Bank with ChatGPT: Few-Shot Text Classification for Finance](https://mlsanity.com/paper/2308.14634/) (2023)
- [Beyond Myopia: Learning from Positive and Unlabeled Data through Holistic Predictive Trends](https://mlsanity.com/paper/2310.04078/) (2023)
- [Quantifying disparities in intimate partner violence: a machine learning method to correct for underreporting](https://mlsanity.com/paper/2110.04133/) (2021)
- [A General Framework for Learning from Weak Supervision](https://mlsanity.com/paper/2402.01922/) (2024)
- [Self-paced Resistance Learning against Overfitting on Noisy Labels](https://mlsanity.com/paper/2105.03059/) (2021)
- [Data Optimization in Deep Learning: A Survey](https://mlsanity.com/paper/2310.16499/) (2023)
- [A Survey of Dataset Refinement for Problems in Computer Vision Datasets](https://mlsanity.com/paper/2210.11717/) (2022)
- [Unified Risk Analysis for Weakly Supervised Learning](https://mlsanity.com/paper/2309.08216/) (2023)
- [Dens-PU: PU Learning with Density-Based Positive Labeled Augmentation](https://mlsanity.com/paper/2303.11848/) (2023)
- [Making Binary Classification from Multiple Unlabeled Datasets Almost Free of Supervision](https://mlsanity.com/paper/2306.07036/) (2023)
- [Multi-class Label Noise Learning via Loss Decomposition and Centroid Estimation](https://mlsanity.com/paper/2203.10858/) (2022)
- [Learning From How Humans Correct](https://mlsanity.com/paper/2102.00225/) (2021)

## Abstract (arXiv)

Noisy PN learning is the problem of binary classification when training examples may be mislabeled (flipped) uniformly with noise rate rho1 for positive examples and rho0 for negative examples. We propose Rank Pruning (RP) to solve noisy PN learning and the open problem of estimating the noise rates, i.e. the fraction of wrong positive and negative labels. Unlike prior solutions, RP is time-efficient and general, requiring O(T) for any unrestricted choice of probabilistic classifier with T fitting time. We prove RP has consistent noise estimation and equivalent expected risk as learning with uncorrupted labels in ideal conditions, and derive closed-form solutions when conditions are non-ideal. RP achieves state-of-the-art noise estimation and F1, error, and AUC-PR for both MNIST and CIFAR datasets, regardless of the amount of noise and performs similarly impressively when a large portion of training examples are noise drawn from a third distribution. To highlight, RP with a CNN classifier can predict if an MNIST digit is a "one"or "not" with only 0.25% error, and 0.46 error across all digits, even when 50% of positive examples are mislabeled and 50% of observed positive labels are mislabeled negative examples.