# Learning with Noisy Labels via Sparse Regularization

arXiv: https://arxiv.org/abs/2108.00192 · HTML: https://mlsanity.com/paper/2108.00192/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 32 references that are in the map.
- First posted on arXiv 31 Jul 2021.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [WebVision Database: Visual Learning and Understanding from Web Data](https://mlsanity.com/paper/1708.02862/) (2017)
- [Normalized Loss Functions for Deep Learning with Noisy Labels](https://mlsanity.com/paper/2006.13554/) (2020)
- [Deep learning with noisy labels: exploring techniques and remedies in medical image analysis](https://mlsanity.com/paper/1912.02911/) (2019)
- [Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks](https://mlsanity.com/paper/1903.11680/) (2019)
- [NLNL: Negative Learning for Noisy Labels](https://mlsanity.com/paper/1908.07387/) (2019)
- [A Survey of Label-noise Representation Learning: Past, Present and Future](https://mlsanity.com/paper/2011.04406/) (2020)
- [SoftSort: A Continuous Relaxation for the argsort Operator](https://mlsanity.com/paper/2006.16038/) (2020)
- [DaST: Data-free Substitute Training for Adversarial Attacks](https://mlsanity.com/paper/2003.12703/) (2020)
- [On Symmetric Losses for Learning from Corrupted Labels](https://mlsanity.com/paper/1901.09314/) (2019)
- [Asymmetric Loss Functions for Learning with Noisy Labels](https://mlsanity.com/paper/2106.03110/) (2021)

## Built on by (most cited first, 30 of 39)

- [Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labeling](https://mlsanity.com/paper/2111.12698/) (2021)
- [Towards Robust Adaptive Object Detection under Noisy Annotations](https://mlsanity.com/paper/2204.02620/) (2022)
- [SILT: Shadow-aware Iterative Label Tuning for Learning to Detect Shadows from Noisy Labels](https://mlsanity.com/paper/2308.12064/) (2023)
- [Scalable Penalized Regression for Noise Detection in Learning with Noisy Labels](https://mlsanity.com/paper/2203.07788/) (2022)
- [LapSum -- One Method to Differentiate Them All: Ranking, Sorting and Top-k Selection](https://mlsanity.com/paper/2503.06242/) (2025)
- [L2B: Learning to Bootstrap Robust Models for Combating Label Noise](https://mlsanity.com/paper/2202.04291/) (2022)
- [Instance-dependent Label Distribution Estimation for Learning with Label Noise](https://mlsanity.com/paper/2212.08380/) (2022)
- [Learning with Noisy labels via Self-supervised Adversarial Noisy Masking](https://mlsanity.com/paper/2302.06805/) (2023)
- [Mitigating Label Noise through Data Ambiguation](https://mlsanity.com/paper/2305.13764/) (2023)
- [Dynamics-Aware Loss for Learning with Label Noise](https://mlsanity.com/paper/2303.11562/) (2023)
- [Sample Prior Guided Robust Model Learning to Suppress Noisy Labels](https://mlsanity.com/paper/2112.01197/) (2021)
- [CNLL: A Semi-supervised Approach For Continual Noisy Label Learning](https://mlsanity.com/paper/2204.09881/) (2022)
- [Joint Class-Affinity Loss Correction for Robust Medical Image Segmentation with Noisy Labels](https://mlsanity.com/paper/2206.07994/) (2022)
- [NoisywikiHow: A Benchmark for Learning with Real-world Noisy Labels in Natural Language Processing](https://mlsanity.com/paper/2305.10709/) (2023)
- [Tackling Noisy Labels with Network Parameter Additive Decomposition](https://mlsanity.com/paper/2403.13241/) (2024)
- [Mid-level Representation Enhancement and Graph Embedded Uncertainty Suppressing for Facial Expression Recognition](https://mlsanity.com/paper/2207.13235/) (2022)
- [Learning with Imbalanced Noisy Data by Preventing Bias in Sample Selection](https://mlsanity.com/paper/2402.11242/) (2024)
- [ε-Softmax: Approximating One-Hot Vectors for Mitigating Label Noise](https://mlsanity.com/paper/2508.02387/) (2025)
- [Unleashing the Potential of Regularization Strategies in Learning with Noisy Labels](https://mlsanity.com/paper/2307.05025/) (2023)
- [Rethinking the impact of noisy labels in graph classification: A utility and privacy perspective](https://mlsanity.com/paper/2406.07314/) (2024)
- [Prototype-Anchored Learning for Learning with Imperfect Annotations](https://mlsanity.com/paper/2206.11602/) (2022)
- [Truncate-Split-Contrast: A Framework for Learning from Mislabeled Videos](https://mlsanity.com/paper/2212.13495/) (2022)
- [SANTA: Separate Strategies for Inaccurate and Incomplete Annotation Noise in Distantly-Supervised Named Entity Recognition](https://mlsanity.com/paper/2305.04076/) (2023)
- [Robust Learning of Diffusion Models with Extremely Noisy Conditions](https://mlsanity.com/paper/2510.10149/) (2025)
- [Do Deep Neural Networks Always Perform Better When Eating More Data?](https://mlsanity.com/paper/2205.15187/) (2022)
- [Knockoffs-SPR: Clean Sample Selection in Learning with Noisy Labels](https://mlsanity.com/paper/2301.00545/) (2023)
- [SplitAMC: Split Learning for Robust Automatic Modulation Classification](https://mlsanity.com/paper/2304.12200/) (2023)
- [LaplaceConfidence: a Graph-based Approach for Learning with Noisy Labels](https://mlsanity.com/paper/2307.16614/) (2023)
- [INSIGHT: Explainable Weakly-Supervised Medical Image Analysis](https://mlsanity.com/paper/2412.02012/) (2024)
- [Pushing One Pair of Labels Apart Each Time in Multi-Label Learning: From Single Positive to Full Labels](https://mlsanity.com/paper/2302.14695/) (2023)

## Abstract (arXiv)

Learning with noisy labels is an important and challenging task for training accurate deep neural networks. Some commonly-used loss functions, such as Cross Entropy (CE), suffer from severe overfitting to noisy labels. Robust loss functions that satisfy the symmetric condition were tailored to remedy this problem, which however encounter the underfitting effect. In this paper, we theoretically prove that any loss can be made robust to noisy labels by restricting the network output to the set of permutations over a fixed vector. When the fixed vector is one-hot, we only need to constrain the output to be one-hot, which however produces zero gradients almost everywhere and thus makes gradient-based optimization difficult. In this work, we introduce the sparse regularization strategy to approximate the one-hot constraint, which is composed of network output sharpening operation that enforces the output distribution of a network to be sharp and the ℓp-norm (p≤ 1) regularization that promotes the network output to be sparse. This simple approach guarantees the robustness of arbitrary loss functions while not hindering the fitting ability. Experimental results demonstrate that our method can significantly improve the performance of commonly-used loss functions in the presence of noisy labels and class imbalance, and outperform the state-of-the-art methods. The code is available at https://github.com/hitcszx/lnlsr.