mlsanityData to 14 Oct 2026

Adversarial Training Should Be Cast as a Non-Zero-Sum Game

arXiv 2306.11035 ↗First posted 19 Jun 2023cs.LG math.OC stat.MLcited by 13

Not on the map. Why?

Abstract (arXiv)

One prominent approach toward resolving the adversarial vulnerability of deep neural networks is the two-player zero-sum paradigm of adversarial training, in which predictors are trained against adversarially chosen perturbations of data. Despite the promise of this approach, algorithms based on this paradigm have not engendered sufficient levels of robustness and suffer from pathological behavior like robust overfitting. To understand this shortcoming, we first show that the commonly used surrogate-based relaxation used in adversarial training algorithms voids all guarantees on the robustness of trained classifiers. The identification of this pitfall informs a novel non-zero-sum bilevel formulation of adversarial training, wherein each player optimizes a different objective function. Our formulation yields a simple algorithmic framework that matches and in some cases outperforms state-of-the-art attacks, attains comparable levels of robustness to standard adversarial training algorithms, and does not suffer from robust overfitting.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 59 of its 72 references that are in the map. First posted on arXiv 19 Jun 2023.

Builds on

59 of 72 references.

References · 59

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  5. 2013Intriguing properties of neural networkscited by 5,155
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  7. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  8. 2019Invariant Risk Minimizationcited by 1,854
  9. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  10. 2016Adversarial examples in the physical worldcited by 1,769
  11. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  12. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
38 more
  1. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  2. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  3. 2018Robustness May Be at Odds with Accuracycited by 880
  4. 2020Fast is better than free: Revisiting adversarial trainingcited by 727
  5. 2017Evasion Attacks against Machine Learning at Test Timecited by 689
  6. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 661
  7. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 567
  8. 2020Overfitting in adversarially robust deep learningcited by 506
  9. 2019On Evaluating Adversarial Robustnesscited by 447
  10. 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 343
  11. 2018Adversarially Robust Generalization Requires More Datacited by 318
  12. 2020An Investigation of Why Overparameterization Exacerbates Spurious Correlationscited by 310
  13. 2021Improving Robustness using Generated Datacited by 234
  14. 2018Adversarial Logit Pairingcited by 224
  15. 2020No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problemscited by 209
  16. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 208
  17. 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 204
  18. 2018Disentangling Adversarial Robustness and Generalizationcited by 172
  19. 2020BREEDS: Benchmarks for Subpopulation Shiftcited by 155
  20. 2020Perceptual Adversarial Robustness: Defense Against Unseen Threat Modelscited by 137
  21. 2019Robust Learning with Jacobian Regularizationcited by 129
  22. 2022Robustness and Accuracy Could Be Reconcilable by (Proper) Definitioncited by 111
  23. 2021Model-Based Domain Generalizationcited by 104
  24. 2020Precise Tradeoffs in Adversarial Training for Linear Regressioncited by 77
  25. 2021Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimizationcited by 76
  26. 2020Adversarial Vertex Mixup: Toward Better Adversarially Robust Generalizationcited by 73
  27. 2015Learning with a Strong Adversarycited by 68
  28. 2022Probable Domain Generalization via Quantile Risk Minimizationcited by 66
  29. 2021Exploring Memorization in Adversarial Trainingcited by 55
  30. 2022Understanding Robust Overfitting of Adversarial Training and Beyondcited by 51
  31. 2021Calibration and Consistency of Adversarial Surrogate Lossescited by 47
  32. 2019An Alternative Surrogate Loss for PGD-based Adversarial Testingcited by 45
  33. 2019Jacobian Adversarially Regularized Networks for Robustnesscited by 40
  34. 2020Provable tradeoffs in adversarially robust classificationcited by 40
  35. 2021Adversarial Robustness with Semi-Infinite Constrained Learningcited by 39
  36. 2022Probabilistically Robust Learning: Balancing Average- and Worst-case Performancecited by 37
  37. 2020Calibrated Surrogate Losses for Adversarially Robust Classificationcited by 36
  38. 2022Revisiting Residual Networks for Adversarial Robustness: An Architectural Perspectivecited by 28

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 5; 2025: 4.

13 papers.

Sort:

By area:

  1. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 469
  2. 2024Rethinking Machine Unlearning for Large Language Modelscited by 257
  3. 2023An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learningcited by 41
  4. 2023Near-Optimal Nonconvex-Strongly-Convex Bilevel Optimization with Fully First-Order Oraclescited by 17
  5. 2024Rethinking Robustness Assessment: Adversarial Attacks on Learning-based Quadrupedal Locomotion Controllerscited by 14
  6. 2025Adversarial Training for Defense Against Label Poisoning Attackscited by 6
  7. 2025Enhancing Adversarial Robustness with Conformal Prediction: A Framework for Guaranteed Model Reliabilitycited by 6
  8. 2023Addressing GAN Training Instabilities via Tunable Classification Lossescited by 5
  9. 2024Extreme Miscalibration and the Illusion of Adversarial Robustnesscited by 4
  10. 2024Improving SAM Requires Rethinking its Optimization Formulationcited by 4
  11. 2025Machine Unlearning Meets Adversarial Robustness via Constrained Interventions on LLMscited by 3
  12. 2024Adversarial Consistency and the Uniqueness of the Adversarial Bayes Classifiercited by 2
1 more
  1. 2025Decoupled SGDA for Games with Intermittent Strategy Communicationcited by 0