# Robustness of Accuracy Metric and its Inspirations in Learning with Noisy Labels

arXiv: https://arxiv.org/abs/2012.04193 · HTML: https://mlsanity.com/paper/2012.04193/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Dec 2020.

## Built on by (most cited first, 20 of 20)

- [Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization](https://mlsanity.com/paper/2102.02414/) (2021)
- [Active label cleaning for improved dataset quality under resource constraints](https://mlsanity.com/paper/2109.00574/) (2021)
- [Early Stopping Against Label Noise Without Validation Data](https://mlsanity.com/paper/2502.07551/) (2025)
- [Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrection](https://mlsanity.com/paper/2111.14932/) (2021)
- [Centrality and Consistency: Two-Stage Clean Samples Identification for Learning with Instance-Dependent Noisy Labels](https://mlsanity.com/paper/2207.14476/) (2022)
- [Searching for Robustness: Loss Learning for Noisy Classification Tasks](https://mlsanity.com/paper/2103.00243/) (2021)
- [Tackling Instance-Dependent Label Noise via a Universal Probabilistic Model](https://mlsanity.com/paper/2101.05467/) (2021)
- [Learning to Rectify for Robust Learning with Noisy Labels](https://mlsanity.com/paper/2111.04239/) (2021)
- [Noisy Label Processing for Classification: A Survey](https://mlsanity.com/paper/2404.04159/) (2024)
- [Label Noise-Robust Learning using a Confidence-Based Sieving Strategy](https://mlsanity.com/paper/2210.05330/) (2022)
- [Learning with Group Noise](https://mlsanity.com/paper/2103.09468/) (2021)
- [Imbalanced Medical Image Segmentation with Pixel-dependent Noisy Labels](https://mlsanity.com/paper/2501.06678/) (2025)
- [A Study on the Impact of Data Augmentation for Training Convolutional Neural Networks in the Presence of Noisy Labels](https://mlsanity.com/paper/2208.11176/) (2022)
- [Leveraging Unlabeled Data to Track Memorization](https://mlsanity.com/paper/2212.04461/) (2022)
- [URL: Combating Label Noise for Lung Nodule Malignancy Grading](https://mlsanity.com/paper/2308.08772/) (2023)
- [Is there a Trojan! : Literature survey and critical evaluation of the latest ML based modern intrusion detection systems in IoT environments](https://mlsanity.com/paper/2310.10778/) (2023)
- [Set a Thief to Catch a Thief: Combating Label Noise through Noisy Meta Learning](https://mlsanity.com/paper/2502.16104/) (2025)
- [crowd-hpo: Realistic Hyperparameter Optimization and Benchmarking for Learning from Crowds with Noisy Labels](https://mlsanity.com/paper/2504.09085/) (2025)
- [Multilingual Machine Translation with Quantum Encoder Decoder Attention-based Convolutional Variational Circuits](https://mlsanity.com/paper/2505.09407/) (2025)
- [LiLAW: Lightweight Learnable Adaptive Weighting to Learn Sample Difficulty & Improve Noisy Training](https://mlsanity.com/paper/2509.20786/) (2025)

## Abstract (arXiv)

For multi-class classification under class-conditional label noise, we prove that the accuracy metric itself can be robust. We concretize this finding's inspiration in two essential aspects: training and validation, with which we address critical issues in learning with noisy labels. For training, we show that maximizing training accuracy on sufficiently many noisy samples yields an approximately optimal classifier. For validation, we prove that a noisy validation set is reliable, addressing the critical demand of model selection in scenarios like hyperparameter-tuning and early stopping. Previously, model selection using noisy validation samples has not been theoretically justified. We verify our theoretical results and additional claims with extensive experiments. We show characterizations of models trained with noisy labels, motivated by our theoretical results, and verify the utility of a noisy validation set by showing the impressive performance of a framework termed noisy best teacher and student (NTS). Our code is released.