# Boosting Co-teaching with Compression Regularization for Label Noise

arXiv: https://arxiv.org/abs/2104.13766 · HTML: https://mlsanity.com/paper/2104.13766/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 30 references that are in the map.
- First posted on arXiv 28 Apr 2021.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [How does Disagreement Help Generalization against Label Corruption?](https://mlsanity.com/paper/1901.04215/) (2019)
- [Joint Optimization Framework for Learning with Noisy Labels](https://mlsanity.com/paper/1803.11364/) (2018)
- [Combating noisy labels by agreement: A joint training method with co-regularization](https://mlsanity.com/paper/2003.02752/) (2020)
- [WebVision Database: Visual Learning and Understanding from Web Data](https://mlsanity.com/paper/1708.02862/) (2017)
- [Decoupling "when to update" from "how to update"](https://mlsanity.com/paper/1706.02613/) (2017)
- [Probabilistic End-to-end Noise Correction for Learning with Noisy Labels](https://mlsanity.com/paper/1903.07788/) (2019)
- [Dimensionality-Driven Learning with Noisy Labels](https://mlsanity.com/paper/1806.02612/) (2018)
- [Learning to Learn from Noisy Labeled Data](https://mlsanity.com/paper/1812.05214/) (2018)
- [Learning with Feature-Dependent Label Noise: A Progressive Approach](https://mlsanity.com/paper/2103.07756/) (2021)
- [Learning Ordered Representations with Nested Dropout](https://mlsanity.com/paper/1402.0915/) (2014)
- [Learning Sparse Networks Using Targeted Dropout](https://mlsanity.com/paper/1905.13678/) (2019)
- [A Semi-Supervised Two-Stage Approach to Learning from Noisy Labels](https://mlsanity.com/paper/1802.02679/) (2018)

## Built on by (most cited first, 30 of 31)

- [ProMix: Combating Label Noise via Maximizing Clean Sample Utility](https://mlsanity.com/paper/2207.10276/) (2022)
- [Textual Knowledge Matters: Cross-Modality Co-Teaching for Generalized Visual Class Discovery](https://mlsanity.com/paper/2403.07369/) (2024)
- [SELC: Self-Ensemble Label Correction Improves Learning with Noisy Labels](https://mlsanity.com/paper/2205.01156/) (2022)
- [Neighborhood Collective Estimation for Noisy Label Identification and Correction](https://mlsanity.com/paper/2208.03207/) (2022)
- [Jigsaw-ViT: Learning Jigsaw Puzzles in Vision Transformer](https://mlsanity.com/paper/2207.11971/) (2022)
- [Identifying Hard Noise in Long-Tailed Sample Distribution](https://mlsanity.com/paper/2207.13378/) (2022)
- [Two Wrongs Don't Make a Right: Combating Confirmation Bias in Learning with Label Noise](https://mlsanity.com/paper/2112.02960/) (2021)
- [Instance-Dependent Noisy Label Learning via Graphical Modelling](https://mlsanity.com/paper/2209.00906/) (2022)
- [CLIPCleaner: Cleaning Noisy Labels with CLIP](https://mlsanity.com/paper/2408.10012/) (2024)
- [Scalable Penalized Regression for Noise Detection in Learning with Noisy Labels](https://mlsanity.com/paper/2203.07788/) (2022)
- [SSR: An Efficient and Robust Framework for Learning with Unknown Label Noise](https://mlsanity.com/paper/2111.11288/) (2021)
- [L2B: Learning to Bootstrap Robust Models for Combating Label Noise](https://mlsanity.com/paper/2202.04291/) (2022)
- [Learning with Noisy labels via Self-supervised Adversarial Noisy Masking](https://mlsanity.com/paper/2302.06805/) (2023)
- [Unsupervised Learning under Latent Label Shift](https://mlsanity.com/paper/2207.13179/) (2022)
- [Compressing Features for Learning with Noisy Labels](https://mlsanity.com/paper/2206.13140/) (2022)
- [ProSelfLC: Progressive Self Label Correction Towards A Low-Temperature Entropy State](https://mlsanity.com/paper/2207.00118/) (2022)
- [Learning with Noisy Labels over Imbalanced Subpopulations](https://mlsanity.com/paper/2211.08722/) (2022)
- [CoDiM: Learning with Noisy Labels via Contrastive Semi-Supervised Learning](https://mlsanity.com/paper/2111.11652/) (2021)
- [ANNE: Adaptive Nearest Neighbors and Eigenvector-based Sample Selection for Robust Learning with Noisy Labels](https://mlsanity.com/paper/2411.01613/) (2024)
- [CNTN: Cyclic Noise-tolerant Network for Gait Recognition](https://mlsanity.com/paper/2210.06910/) (2022)
- [Robust Object Detection in Remote Sensing Imagery with Noisy and Sparse Geo-Annotations (Full Version)](https://mlsanity.com/paper/2210.12989/) (2022)
- [Asymmetric Co-teaching with Multi-view Consensus for Noisy Label Learning](https://mlsanity.com/paper/2301.01143/) (2023)
- [Towards the Identifiability in Noisy Label Learning: A Multinomial Mixture Modelling Approach](https://mlsanity.com/paper/2301.01405/) (2023)
- [PASS: Peer-Agreement based Sample Selection for training with Noisy Labels](https://mlsanity.com/paper/2303.10802/) (2023)
- [Bridging Generative and Discriminative Noisy-Label Learning via Direction-Agnostic EM Formulation](https://mlsanity.com/paper/2308.01184/) (2023)
- [Feature Noise Boosts DNN Generalization under Label Noise](https://mlsanity.com/paper/2308.01609/) (2023)
- [Collaboration of Teachers for Semi-supervised Object Detection](https://mlsanity.com/paper/2405.13374/) (2024)
- [A Multi-module Robust Method for Transient Stability Assessment against False Label Injection Cyberattacks](https://mlsanity.com/paper/2406.06744/) (2024)
- [CLID-MU: Cross-Layer Information Divergence Based Meta Update Strategy for Learning with Noisy Labels](https://mlsanity.com/paper/2507.11807/) (2025)
- [Learning to Clean: Reinforcement Learning for Noisy Label Correction](https://mlsanity.com/paper/2511.19808/) (2025)

## Abstract (arXiv)

In this paper, we study the problem of learning image classification models in the presence of label noise. We revisit a simple compression regularization named Nested Dropout. We find that Nested Dropout, though originally proposed to perform fast information retrieval and adaptive data compression, can properly regularize a neural network to combat label noise. Moreover, owing to its simplicity, it can be easily combined with Co-teaching to further boost the performance. Our final model remains simple yet effective: it achieves comparable or even better performance than the state-of-the-art approaches on two real-world datasets with label noise which are Clothing1M and ANIMAL-10N. On Clothing1M, our approach obtains 74.9% accuracy which is slightly better than that of DivideMix. On ANIMAL-10N, we achieve 84.1% accuracy while the best public result by PLC is 83.4%. We hope that our simple approach can be served as a strong baseline for learning with label noise. Our implementation is available at https://github.com/yingyichen-cyy/Nested-Co-teaching.