mlsanityData to 7 Oct 2026

Improving robustness to corruptions with multiplicative weight perturbations

arXiv 2406.16540 ↗First posted 24 Jun 2024cs.CV cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) excel on clean images but struggle with corrupted ones. Incorporating specific corruptions into the data augmentation pipeline can improve robustness to those corruptions but may harm performance on clean images and other types of distortion. In this paper, we introduce an alternative approach that improves the robustness of DNNs to a wide range of corruptions without compromising accuracy on clean images. We first demonstrate that input perturbations can be mimicked by multiplicative perturbations in the weight space. Leveraging this, we propose Data Augmentation via Multiplicative Perturbation (DAMP), a training method that optimizes DNNs under random multiplicative weight perturbations. We also examine the recently proposed Adaptive Sharpness-Aware Minimization (ASAM) and show that it optimizes DNNs under adversarial multiplicative weight perturbations. Experiments on image classification datasets (CIFAR-10/100, TinyImageNet and ImageNet) and neural network architectures (ResNet50, ViT-S/16, ViT-B/16) show that DAMP enhances model generalization performance in the presence of corruptions across different settings. Notably, DAMP is able to train a ViT-S/16 on ImageNet from scratch, reaching the top-1 error of 23.7% which is comparable to ResNet50 without extensive data augmentations.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 38 of its 50 references that are in the map. First posted on arXiv 24 Jun 2024.

Builds on

38 of 50 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  6. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  7. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  8. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  9. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  10. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  11. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  12. 2016Concrete Problems in AI Safetycited by 1,582
26 more
  1. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  2. 2019Natural Adversarial Examplescited by 1,401
  3. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  4. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  5. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  6. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  7. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  8. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  9. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
  10. 2020Improving robustness against common corruptions by covariate shift adaptationcited by 427
  11. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 332
  12. 2021ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networkscited by 293
  13. 2016Revisiting Batch Normalization For Practical Domain Adaptationcited by 285
  14. 2021When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentationscited by 259
  15. 2018Generalisation in humans and deep neural networkscited by 237
  16. 2020Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shiftcited by 235
  17. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
  18. 2017Multiplicative Normalizing Flows for Variational Bayesian Neural Networkscited by 157
  19. 2019Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentationcited by 105
  20. 2021On Interaction Between Augmentations and Corruptions in Natural Corruption Robustnesscited by 98
  21. 2022Better plain ViT baselines for ImageNet-1kcited by 97
  22. 2020Revisiting Batch Normalization for Improving Corruption Robustnesscited by 55
  23. 2023ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classificationcited by 33
  24. 2021Dangers of Bayesian Model Averaging under Covariate Shiftcited by 31
  25. 2024ImageNet-D: Benchmarking Neural Network Robustness on Diffusion Synthetic Objectcited by 22
  26. 2022Tackling covariate shift with node-based Bayesian neural networkscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 5.

7 papers.

  1. 2026C-RADIOv4 (Tech Report)cited by 10
  2. 2025Log-Normal Multiplicative Dynamics for Stable Low-Precision Training of Large Networkscited by 2
  3. 2025From Code to Field: Evaluating the Robustness of Convolutional Neural Networks for Disease Diagnosis in Mango Leavescited by 0
  4. 2026Boosting Robust AIGI Detection with LoRA-based Pairwise Trainingcited by 0
  5. 2026Robustness Emerges Early in Training Dynamics, but Is Not Preservedcited by 0
  6. 2026Suppress and Diversify: Refining Robust Pathways for Corruption Robustnesscited by 0
  7. 2026Learning Perturbation Robust Policies for LLM Agents with Stable Optimizationcited by 0