mlsanityData to 7 Oct 2026

Multiplicative Reweighting for Robust Neural Network Optimization

arXiv 2102.12192 ↗First posted 24 Feb 2021cs.LG cs.CRcited by 4

Not on the map. Why?

Abstract (arXiv)

Neural networks are widespread due to their powerful performance. Yet, they degrade in the presence of noisy labels at training time. Inspired by the setting of learning with expert advice, where multiplicative weights (MW) updates were recently shown to be robust to moderate data corruptions in expert advice, we propose to use MW for reweighting examples during neural networks optimization. We theoretically establish the convergence of our method when used with gradient descent and prove its advantages in 1d cases. We then validate empirically our findings for the general case by showing that MW improves neural networks' accuracy in the presence of label noise on CIFAR-10, CIFAR-100 and Clothing1M. We also show the impact of our approach on adversarial robustness.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 93 of its 137 references that are in the map. First posted on arXiv 24 Feb 2021.

Builds on

93 of 137 references.

References · 93

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  6. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  7. 2013Intriguing properties of neural networkscited by 5,114
  8. 2016Wide Residual Networkscited by 3,057
  9. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  10. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  11. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  12. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
38 more
  1. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  2. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  3. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  4. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
  5. 2016Adversarial Machine Learning at Scalecited by 934
  6. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  7. 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
  8. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  9. 2016Universal adversarial perturbationscited by 774
  10. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
  11. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 691
  12. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  13. 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
  14. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  15. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
  16. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
  17. 2019Adversarial Training for Free!cited by 638
  18. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 580
  19. 2014Convex Optimization: Algorithms and Complexitycited by 569
  20. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
  21. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 499
  22. 2017Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networkscited by 484
  23. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 441
  24. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 423
  25. 2019Unlabeled Data Improves Adversarial Robustnesscited by 415
  26. 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 414
  27. 2019Unsupervised Label Noise Modeling and Loss Correctioncited by 327
  28. 2018Feature Denoising for Improving Adversarial Robustnesscited by 312
  29. 2018Joint Optimization Framework for Learning with Noisy Labelscited by 293
  30. 2020Combating noisy labels by agreement: A joint training method with co-regularizationcited by 289
  31. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 288
  32. 2017WebVision Database: Visual Learning and Understanding from Web Datacited by 262
  33. 2017Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradientscited by 236
  34. 2017Decoupling "when to update" from "how to update"cited by 230
  35. 2018Adversarial Logit Pairingcited by 224
  36. 2018Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noisecited by 216
  37. 2018Adversarial Robustness Toolbox v1.0.0cited by 214
  38. 2019Are Anchor Points Really Indispensable in Label-Noise Learning?cited by 208

50 most cited of 93 All 93 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 1.

4 papers.

  1. 2022To Aggregate or Not? Learning with Separate Noisy Labelscited by 29
  2. 2022Detecting Label Errors by using Pre-Trained Language Modelscited by 16
  3. 2021Constrained Instance and Class Reweighting for Robust Learning under Label Noisecited by 12
  4. 2023SLaM: Student-Label Mixing for Distillation with Unlabeled Examplescited by 9