# Constrained Instance and Class Reweighting for Robust Learning under Label Noise

arXiv: https://arxiv.org/abs/2111.05428 · HTML: https://mlsanity.com/paper/2111.05428/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 49 references that are in the map.
- First posted on arXiv 9 Nov 2021.

## Builds on (most cited first, 29 of 29)

- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [Early-Learning Regularization Prevents Memorization of Noisy Labels](https://mlsanity.com/paper/2007.00151/) (2020)
- [Unsupervised Label Noise Modeling and Loss Correction](https://mlsanity.com/paper/1904.11238/) (2019)
- [Training Deep Neural Networks on Noisy Labels with Bootstrapping](https://mlsanity.com/paper/1412.6596/) (2014)
- [Combating noisy labels by agreement: A joint training method with co-regularization](https://mlsanity.com/paper/2003.02752/) (2020)
- [Normalized Loss Functions for Deep Learning with Noisy Labels](https://mlsanity.com/paper/2006.13554/) (2020)
- [Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise](https://mlsanity.com/paper/1802.05300/) (2018)
- [Does label smoothing mitigate label noise?](https://mlsanity.com/paper/2003.02819/) (2020)
- [A Rewriting System for Convex Optimization Problems](https://mlsanity.com/paper/1709.04494/) (2017)
- [Large-Scale Methods for Distributionally Robust Optimization](https://mlsanity.com/paper/2010.05893/) (2020)
- [SELF: Learning to Filter Noisy Labels with Self-Ensembling](https://mlsanity.com/paper/1910.01842/) (2019)
- [Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks](https://mlsanity.com/paper/1903.11680/) (2019)
- [Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates](https://mlsanity.com/paper/1910.03231/) (2019)
- [A Survey of Label-noise Representation Learning: Past, Present and Future](https://mlsanity.com/paper/2011.04406/) (2020)
- [Robust Bi-Tempered Logistic Loss Based on Bregman Divergences](https://mlsanity.com/paper/1906.03361/) (2019)
- [Correlated Input-Dependent Label Noise in Large-Scale Image Classification](https://mlsanity.com/paper/2105.10305/) (2021)
- [Exponentiated Gradient Reweighting for Robust Training Under Label Noise and Beyond](https://mlsanity.com/paper/2104.01493/) (2021)
- [Auxiliary Image Regularization for Deep CNNs with Noisy Labels](https://mlsanity.com/paper/1511.07069/) (2015)
- [Multiplicative Reweighting for Robust Neural Network Optimization](https://mlsanity.com/paper/2102.12192/) (2021)

## Built on by (most cited first, 12 of 12)

- [To Aggregate or Not? Learning with Separate Noisy Labels](https://mlsanity.com/paper/2206.07181/) (2022)
- [Detecting Label Errors by using Pre-Trained Language Models](https://mlsanity.com/paper/2205.12702/) (2022)
- [Distributionally Robust Post-hoc Classifiers under Prior Shifts](https://mlsanity.com/paper/2309.08825/) (2023)
- [Weighted Distillation with Unlabeled Examples](https://mlsanity.com/paper/2210.06711/) (2022)
- [SLaM: Student-Label Mixing for Distillation with Unlabeled Examples](https://mlsanity.com/paper/2302.03806/) (2023)
- [Dynamics-Aware Loss for Learning with Label Noise](https://mlsanity.com/paper/2303.11562/) (2023)
- [Solving Inverse Problems with NerfGANs](https://mlsanity.com/paper/2112.09061/) (2021)
- [Stochastic Re-weighted Gradient Descent via Distributionally Robust Optimization](https://mlsanity.com/paper/2306.09222/) (2023)
- [When VLMs Meet Image Classification: Test Sets Renovation via Missing Label Identification](https://mlsanity.com/paper/2505.16149/) (2025)
- [Adaptive Deviation Learning for Visual Anomaly Detection with Data Contamination](https://mlsanity.com/paper/2411.09558/) (2024)
- [LiLAW: Lightweight Learnable Adaptive Weighting to Learn Sample Difficulty & Improve Noisy Training](https://mlsanity.com/paper/2509.20786/) (2025)
- [Multi-Cue Anomaly Detection and Localization under Data Contamination](https://mlsanity.com/paper/2601.22913/) (2026)

## Abstract (arXiv)

Deep neural networks have shown impressive performance in supervised learning, enabled by their ability to fit well to the provided training data. However, their performance is largely dependent on the quality of the training data and often degrades in the presence of noise. We propose a principled approach for tackling label noise with the aim of assigning importance weights to individual instances and class labels. Our method works by formulating a class of constrained optimization problems that yield simple closed form updates for these importance weights. The proposed optimization problems are solved per mini-batch which obviates the need of storing and updating the weights over the full dataset. Our optimization framework also provides a theoretical perspective on existing label smoothing heuristics for addressing label noise (such as label bootstrapping). We evaluate our method on several benchmark datasets and observe considerable performance gains in the presence of label noise.