mlsanityData to 7 Oct 2026

Boosting Adversarial Robustness From The Perspective of Effective Margin Regularization

arXiv 2210.05118 ↗First posted 11 Oct 2022cs.LG cs.CV stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

The adversarial vulnerability of deep neural networks (DNNs) has been actively investigated in the past several years. This paper investigates the scale-variant property of cross-entropy loss, which is the most commonly used loss function in classification tasks, and its impact on the effective margin and adversarial robustness of deep neural networks. Since the loss function is not invariant to logit scaling, increasing the effective weight norm will make the loss approach zero and its gradient vanish while the effective margin is not adequately maximized. On typical DNNs, we demonstrate that, if not properly regularized, the standard training does not learn large effective margins and leads to adversarial vulnerability. To maximize the effective margins and learn a robust DNN, we propose to regularize the effective weight norm during training. Our empirical study on feedforward DNNs demonstrates that the proposed effective margin regularization (EMR) learns large effective margins and boosts the adversarial robustness in both standard and adversarial training. On large-scale models, we show that EMR outperforms basic adversarial training, TRADES and two regularization baselines with substantial improvement. Moreover, when combined with several strong adversarial defense methods (MART and MAIL), our EMR further boosts the robustness.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 44 of its 53 references that are in the map. First posted on arXiv 11 Oct 2022.

Builds on

44 of 53 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2016Wide Residual Networkscited by 3,057
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  7. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  8. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  9. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  10. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  11. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  12. 2016Adversarial Machine Learning at Scalecited by 934
32 more
  1. 2018Robustness May Be at Odds with Accuracycited by 873
  2. 2017SphereFace: Deep Hypersphere Embedding for Face Recognitioncited by 733
  3. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
  4. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  5. 2017Countering Adversarial Images using Input Transformationscited by 517
  6. 2020Overfitting in adversarially robust deep learningcited by 504
  7. 2017Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networkscited by 484
  8. 2017Mitigating Adversarial Effects Through Randomizationcited by 382
  9. 2018Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Modelscited by 362
  10. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
  11. 2019Gradient Descent Maximizes the Margin of Homogeneous Neural Networkscited by 308
  12. 2016Large-Margin Softmax Loss for Convolutional Neural Networkscited by 296
  13. 2018Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionalitycited by 280
  14. 2018Additive Margin Softmax for Face Verificationcited by 275
  15. 2020Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Losscited by 267
  16. 2017PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examplescited by 243
  17. 2017Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradientscited by 236
  18. 2021Improving Robustness using Generated Datacited by 233
  19. 2021Data Augmentation Can Improve Robustnesscited by 193
  20. 2018Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balancedcited by 172
  21. 2018Stochastic Activation Pruning for Robust Adversarial Defensecited by 171
  22. 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 171
  23. 2019Adversarial Robustness through Local Linearizationcited by 146
  24. 2020Boosting Adversarial Training with Hypersphere Embeddingcited by 97
  25. 2020Learnable Boundary Guided Adversarial Trainingcited by 93
  26. 2020Smooth Adversarial Trainingcited by 89
  27. 2017SafetyNet: Detecting and Rejecting Adversarial Examples Robustlycited by 76
  28. 2019The Odds are Odd: A Statistical Test for Detecting Adversarial Examplescited by 75
  29. 2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 73
  30. 2021Probabilistic Margins for Instance Reweighting in Adversarial Trainingcited by 57
  31. 2020Improve Generalization and Robustness of Neural Networks via Weight Scale Shifting Invariant Regularizationscited by 9
  32. 2021Uniform Convergence, Adversarial Spheres and a Simple Remedycited by 8

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2.

4 papers.

  1. 2023TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalizationcited by 18
  2. 2023ODAM: Gradient-based instance-specific visual explanations for object detectioncited by 12
  3. 2024The Pitfalls and Promise of Conformal Inference Under Adversarial Attackscited by 8
  4. 2024TIMA: Text-Image Mutual Awareness for Balancing Zero-Shot Adversarial Robustness and Generalization Abilitycited by 2