# Label-Noise Resistant Learning via Optimal Brain Damage Masking

arXiv: https://arxiv.org/abs/2508.09697 · HTML: https://mlsanity.com/paper/2508.09697/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 56 references that are in the map.
- First posted on arXiv 13 Aug 2025.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [Early-Learning Regularization Prevents Memorization of Noisy Labels](https://mlsanity.com/paper/2007.00151/) (2020)
- [Combating noisy labels by agreement: A joint training method with co-regularization](https://mlsanity.com/paper/2003.02752/) (2020)
- [Normalized Loss Functions for Deep Learning with Noisy Labels](https://mlsanity.com/paper/2006.13554/) (2020)
- [Probabilistic End-to-end Noise Correction for Learning with Noisy Labels](https://mlsanity.com/paper/1903.07788/) (2019)
- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [Robust Training under Label Noise by Over-parameterization](https://mlsanity.com/paper/2202.14026/) (2022)
- [Jo-SRC: A Contrastive Approach for Combating Noisy Labels](https://mlsanity.com/paper/2103.13029/) (2021)
- [UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learning](https://mlsanity.com/paper/2203.14542/) (2022)
- [Convolutional Neural Network Pruning with Structural Redundancy Reduction](https://mlsanity.com/paper/2104.03438/) (2021)
- [A Survey on Kolmogorov-Arnold Network](https://mlsanity.com/paper/2411.06078/) (2024)
- [Detecting Corrupted Labels Without Training a Model to Predict](https://mlsanity.com/paper/2110.06283/) (2021)
- [ProMix: Combating Label Noise via Maximizing Clean Sample Utility](https://mlsanity.com/paper/2207.10276/) (2022)
- [Neighborhood Collective Estimation for Noisy Label Identification and Correction](https://mlsanity.com/paper/2208.03207/) (2022)
- [FC-KAN: Function Combinations in Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2409.01763/) (2024)
- [Searching for Robustness: Loss Learning for Noisy Classification Tasks](https://mlsanity.com/paper/2103.00243/) (2021)
- [Foster Adaptivity and Balance in Learning with Noisy Labels](https://mlsanity.com/paper/2407.02778/) (2024)
- [WebVision Challenge: Visual Learning and Understanding With Web Data](https://mlsanity.com/paper/1705.05640/) (2017)
- [Learning with Imbalanced Noisy Data by Preventing Bias in Sample Selection](https://mlsanity.com/paper/2402.11242/) (2024)
- [Unlabeled Data Improves Fine-Grained Image Zero-shot Classification with Multimodal LLMs](https://mlsanity.com/paper/2506.03195/) (2025)
- [DiscrimLoss: A Universal Loss for Hard Samples and Incorrect Samples Discrimination](https://mlsanity.com/paper/2208.09884/) (2022)
- [Dynamic DropConnect: Enhancing Neural Network Robustness through Adaptive Edge Dropping Strategies](https://mlsanity.com/paper/2502.19948/) (2025)

## Abstract (arXiv)

Noisy labels are inevitable in real-world multimedia applications. Due to the strong memorization capacity of deep neural networks, these noisy labels cause significant performance degradation. Existing noise-robust methods have mainly focused on robust loss functions and sample selection strategies, with comparatively limited exploration of dynamic architectural adaptation. In this paper, we rethink the role of classifier connectivity under label noise. Intuitively, performance degradation stems from the backpropagation of noisy gradients. Since the final classifier layer acts as the primary gateway for this error propagation, selectively discarding redundant connections can restrict the backpropagation pathways of noisy gradients. Consequently, to identify redundant connections, we leverage the seminal Optimal Brain Damage (OBD) theory from model compression, which posits that parameters causing negligible loss perturbation can be removed. Guided by this principle, we show that masking low-activation edges limits the estimated loss perturbation to preserve the model's fitting capacity, while tightening a derived upper bound on noise-induced gradient error. To bridge this theoretical insight with practical training, we propose a novel Selective Edge Masking (SEM) mechanism for the fully connected (FC) layer to enhance noise robustness. It can adaptively retain critical edges for information propagation while suppressing gradient errors caused by noisy labels. As a plug-and-play component, SEM can be seamlessly integrated into various noise-robust methods. Additionally, we validate SEM's applicability by applying it to the newly emerged Kolmogorov-Arnold Network (KAN) employed as a classifier. Extensive evaluations on synthetic and real-world benchmarks demonstrate that our approach achieves state-of-the-art performance.