mlsanityData to 14 Oct 2026

Lower Difficulty and Better Robustness: A Bregman Divergence Perspective for Adversarial Training

arXiv 2208.12511 ↗First posted 26 Aug 2022cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we investigate on improving the adversarial robustness obtained in adversarial training (AT) via reducing the difficulty of optimization. To better study this problem, we build a novel Bregman divergence perspective for AT, in which AT can be viewed as the sliding process of the training data points on the negative entropy curve. Based on this perspective, we analyze the learning objectives of two typical AT methods, i.e., PGD-AT and TRADES, and we find that the optimization process of TRADES is easier than PGD-AT for that TRADES separates PGD-AT. In addition, we discuss the function of entropy in TRADES, and we find that models with high entropy can be better robustness learners. Inspired by the above findings, we propose two methods, i.e., FAIT and MER, which can both not only reduce the difficulty of optimization under the 10-step PGD adversaries, but also provide better robustness. Our work suggests that reducing the difficulty of optimization under the 10-step PGD adversaries is a promising approach for enhancing the adversarial robustness in AT.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 47 references that are in the map. First posted on arXiv 26 Aug 2022.

Builds on

36 of 47 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2017Squeeze-and-Excitation Networkscited by 4,789
  5. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  6. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  7. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
  8. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  9. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,136
  10. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
  11. 2018Robustness May Be at Odds with Accuracycited by 880
  12. 2020Fast is better than free: Revisiting adversarial trainingcited by 727
24 more
  1. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  2. 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 464
  3. 2019Unlabeled Data Improves Adversarial Robustnesscited by 416
  4. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 334
  5. 2018Adversarially Robust Generalization Requires More Datacited by 318
  6. 2019Adversarial Examples Improve Image Recognitioncited by 286
  7. 2020Attacks Which Do Not Kill Training Make Adversarial Learning Strongercited by 243
  8. 2017On Detecting Adversarial Perturbationscited by 232
  9. 2020Understanding and Improving Fast Adversarial Trainingcited by 214
  10. 2020Geometry-aware Instance-reweighted Adversarial Trainingcited by 198
  11. 2018Disentangling Adversarial Robustness and Generalizationcited by 172
  12. 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 172
  13. 2019Are Labels Required for Improving Adversarial Robustness?cited by 163
  14. 2020Understanding and Mitigating the Tradeoff Between Robustness and Accuracycited by 149
  15. 2018Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Modelscited by 135
  16. 2022Robustness and Accuracy Could Be Reconcilable by (Proper) Definitioncited by 111
  17. 2018Universal Adversarial Trainingcited by 87
  18. 2021Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Bettercited by 79
  19. 2018Maximum-Entropy Fine-Grained Classificationcited by 65
  20. 2021Boosting Fast Adversarial Training with Learnable Adversarial Initializationcited by 40
  21. 2019Adversarial Robustness May Be at Odds With Simplicitycited by 39
  22. 2020Towards Understanding Fast Adversarial Trainingcited by 33
  23. 2019Incremental Learning with Maximum Entropy Regularization: Rethinking Forgetting and Intransigencecited by 12
  24. 2022Why Adversarial Training of ReLU Networks Is Difficult?cited by 4

Built on by

Not cited yet.