mlsanityData to 14 Oct 2026

DisturbLabel: Regularizing CNN on the Loss Layer

arXiv 1605.00055 ↗First posted 30 Apr 2016cs.CVcited by 41

Not on the map. Why?

Abstract (arXiv)

During a long period of time we are combating over-fitting in the CNN training process with model regularization, including weight decay, model averaging, data augmentation, etc. In this paper, we present DisturbLabel, an extremely simple algorithm which randomly replaces a part of labels as incorrect values in each iteration. Although it seems weird to intentionally generate incorrect training labels, we show that DisturbLabel prevents the network training from over-fitting by implicitly averaging over exponentially many networks which are trained with different label sets. To the best of our knowledge, DisturbLabel serves as the first work which adds noises on the loss layer. Meanwhile, DisturbLabel cooperates well with Dropout to provide complementary regularization functions. Experiments demonstrate competitive recognition results on several popular image recognition datasets.

Key facts. Cited by 41 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 Apr 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 14; 2022: 9; 2023: 8; 2024: 2; 2025: 5; 2026: 3.

41 papers.

Sort:

By area:

  1. 2021The Devil is in the Margin: Margin-based Label Smoothing for Network Calibrationcited by 91
  2. 2021Open-set Label Noise Can Improve Robustness Against Inherent Label Noisecited by 59
  3. 2021Salient Objects in Cluttercited by 19
  4. 2022Information-Theoretic Bias Reduction via Causal View of Spurious Correlationcited by 15
  5. 2022On the Privacy Effect of Data Enhancement via the Lens of Memorizationcited by 15
  6. 2021Adaptive Label Smoothing To Regularize Large-Scale Graph Trainingcited by 12
  7. 2021Iterative Teaching by Label Synthesiscited by 10
  8. 2021Dense Uncertainty Estimationcited by 9
  9. 2022GeoPointGAN: Synthetic Spatial Data with Local Label Differential Privacycited by 9
  10. 2021NoiLIn: Improving Adversarial Training and Correcting Stereotype of Noisy Labelscited by 8
  11. 2024Relaxed Quantile Regression: Prediction Intervals for Asymmetric Noisecited by 8
  12. 2023DynamicKD: An Effective Knowledge Distillation via Dynamic Entropy Correction-Based Distillation for Gap Optimizingcited by 7
29 more
  1. 2022ProSelfLC: Progressive Self Label Correction Towards A Low-Temperature Entropy Statecited by 5
  2. 2023Enhancing Sample Utilization through Sample Adaptive Augmentation in Semi-Supervised Learningcited by 5
  3. 2022Predicting Out-of-Domain Generalization with Neighborhood Invariancecited by 4
  4. 2022Class-Level Logit Perturbationcited by 4
  5. 2023Rethinking Class Imbalance in Machine Learningcited by 4
  6. 2021Midpoint Regularization: from High Uncertainty Training to Conservative Classificationcited by 3
  7. 2023Model Calibration in Dense Classification with Adaptive Label Perturbationcited by 3
  8. 2025DEAL-300K: Diffusion-based Editing Area Localization with a 300K-Scale Dataset and Frequency-Prompted Baselinecited by 3
  9. 2021Alleviating Noisy-label Effects in Image Classification via Probability Transition Matrixcited by 2
  10. 2022Generalised Co-Salient Object Detectioncited by 2
  11. 2023Implicit Counterfactual Data Augmentation for Robust Learningcited by 2
  12. 2023Mitigating Shortcuts in Language Models with Soft Label Encodingcited by 2
  13. 2026Temporal Difference Learning with Constrained Initial Representationscited by 2
  14. 2021Label Smoothed Embedding Hypothesis for Out-of-Distribution Detectioncited by 1
  15. 2021Bias Loss for Mobile Neural Networkscited by 1
  16. 2022Context-based Virtual Adversarial Training for Text Classification with Noisy Labelscited by 1
  17. 2022Confidence-Aware Paced-Curriculum Learning by Label Smoothing for Surgical Scene Understandingcited by 1
  18. 2023Keep It Simple: CNN Model Complexity Studies for Interference Classification Taskscited by 1
  19. 2025Regularization of ML models for Earth systems by using longer model timestepscited by 1
  20. 2021Self-Supervised Learning from Semantically Imprecise Datacited by 0
  21. 2021Disturbing Target Values for Neural Network Regularizationcited by 0
  22. 2021Altering Backward Pass Gradients improves Convergencecited by 0
  23. 2023Phantom Embeddings: Using Embedding Space for Model Regularization in Deep Neural Networkscited by 0
  24. 2024Improving Time Series Classification with Representation Soft Label Smoothingcited by 0
  25. 2025Intra-class Patch Swap for Self-Distillationcited by 0
  26. 2025Relevance-driven Input Dropout: an Explanation-guided Regularization Techniquecited by 0
  27. 2025Combined Image Data Augmentations diminish the benefits of Adaptive Label Smoothingcited by 0
  28. 2026Less Data Approximates More: Earning Faithful Confidence in High-Stakes Domainscited by 0
  29. 2026Same Target, Different Basins: Hard vs. Soft Labels for Annotator Distributionscited by 0