mlsanityData to 7 Oct 2026

Standard-Deviation-Inspired Regularization for Improving Adversarial Robustness

arXiv 2412.19947 ↗First posted 27 Dec 2024cs.LG cs.AI cs.CRcited by 0

Not on the map. Why?

Abstract (arXiv)

Adversarial Training (AT) has been demonstrated to improve the robustness of deep neural networks (DNNs) against adversarial attacks. AT is a min-max optimization procedure where in adversarial examples are generated to train a more robust DNN. The inner maximization step of AT increases the losses of inputs with respect to their actual classes. The outer minimization involves minimizing the losses on the adversarial examples obtained from the inner maximization. This work proposes a standard-deviation-inspired (SDI) regularization term to improve adversarial robustness and generalization. We argue that the inner maximization in AT is similar to minimizing a modified standard deviation of the model's output probabilities. Moreover, we suggest that maximizing this modified standard deviation can complement the outer minimization of the AT framework. To support our argument, we experimentally show that the SDI measure can be used to craft adversarial examples. Additionally, we demonstrate that combining the SDI regularization term with existing AT variants enhances the robustness of DNNs against stronger attacks, such as CW and Auto-attack, and improves generalization.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 39 of its 52 references that are in the map. First posted on arXiv 27 Dec 2024.

Builds on

39 of 52 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  7. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  8. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  9. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  10. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  11. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  12. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
27 more
  1. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
  2. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
  3. 2019Adversarial Training for Free!cited by 638
  4. 2017Countering Adversarial Images using Input Transformationscited by 517
  5. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
  6. 2018Adversarial Logit Pairingcited by 224
  7. 2018Adversarial Risk and the Dangers of Evaluating Against Weak Attackscited by 221
  8. 2020Understanding and Improving Fast Adversarial Trainingcited by 213
  9. 2020Geometry-aware Instance-reweighted Adversarial Trainingcited by 198
  10. 2018Stochastic Activation Pruning for Robust Adversarial Defensecited by 171
  11. 2020Boosting Adversarial Training with Hypersphere Embeddingcited by 97
  12. 2022LAS-AT: Adversarial Training with Learnable Attack Strategycited by 94
  13. 2021Probabilistic Margins for Instance Reweighting in Adversarial Trainingcited by 57
  14. 2018Improving the Generalization of Adversarial Training with Domain Adaptationcited by 48
  15. 2018Evaluating and Understanding the Robustness of Adversarial Logit Pairingcited by 34
  16. 2023Randomized Adversarial Training via Taylor Expansioncited by 28
  17. 2018Logit Pairing Methods Can Fool Gradient-Based Attackscited by 27
  18. 2019Improving the Robustness of Deep Neural Networks via Adversarial Training with Triplet Losscited by 13
  19. 2022Squeeze Training for Adversarial Robustnesscited by 10
  20. 2024Boosting Adversarial Training via Fisher-Rao Norm-based Regularizationcited by 7
  21. 2023Improving Adversarial Robustness with Hypersphere Embedding and Angular-based Regularizationscited by 5
  22. 2024Extreme Miscalibration and the Illusion of Adversarial Robustnesscited by 4
  23. 2019Improved Adversarial Robustness via Logit Regularization Methodscited by 3
  24. 2020Constraining Logits by Bounded Function for Adversarial Robustnesscited by 3
  25. 2023CAT:Collaborative Adversarial Trainingcited by 2
  26. 2023Vulnerability-Aware Instance Reweighting For Adversarial Trainingcited by 2
  27. 2024Improving Adversarial Training using Vulnerability-Aware Perturbation Budgetcited by 1

Built on by

Not cited yet.