mlsanityData to 14 Oct 2026

Robustness and Accuracy Could Be Reconcilable by (Proper) Definition

arXiv 2202.10103 ↗First posted 21 Feb 2022cs.LG cs.CR stat.MLcited by 111

Not on the map. Why?

Abstract (arXiv)

The trade-off between robustness and accuracy has been widely studied in the adversarial literature. Although still controversial, the prevailing view is that this trade-off is inherent, either empirically or theoretically. Thus, we dig for the origin of this trade-off in adversarial training and find that it may stem from the improperly defined robust error, which imposes an inductive bias of local invariance -- an overcorrection towards smoothness. Given this, we advocate employing local equivariance to describe the ideal behavior of a robust model, leading to a self-consistent robust error named SCORE. By definition, SCORE facilitates the reconciliation between robustness and accuracy, while still handling the worst-case uncertainty via robust optimization. By simply substituting KL divergence with variants of distance metrics, SCORE can be efficiently minimized. Empirically, our models achieve top-rank performance on RobustBench under AutoAttack. Besides, SCORE provides instructive insights for explaining the overfitting phenomenon and semantic input gradients observed on robust models. Code is available at https://github.com/P2333/SCORE.

Key facts. Cited by 111 papers in the map (data through 14 Oct 2026). Builds on 80 of its 108 references that are in the map. First posted on arXiv 21 Feb 2022.

Builds on

80 of 108 references.

References · 80

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  5. 2013Intriguing properties of neural networkscited by 5,155
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  7. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  8. 2016Wide Residual Networkscited by 3,070
  9. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  10. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  11. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  12. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
38 more
  1. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
  2. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
  3. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  4. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  5. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
  6. 2018Robustness May Be at Odds with Accuracycited by 880
  7. 2020Fast is better than free: Revisiting adversarial trainingcited by 727
  8. 2017Evasion Attacks against Machine Learning at Test Timecited by 689
  9. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 661
  10. 2019Adversarial Training for Free!cited by 643
  11. 2020Overfitting in adversarially robust deep learningcited by 506
  12. 2019On Evaluating Adversarial Robustnesscited by 447
  13. 2019Unlabeled Data Improves Adversarial Robustnesscited by 416
  14. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 405
  15. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 384
  16. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  17. 2018Adversarially Robust Generalization Requires More Datacited by 318
  18. 2019Adversarial Examples Improve Image Recognitioncited by 286
  19. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 278
  20. 2020Attacks Which Do Not Kill Training Make Adversarial Learning Strongercited by 243
  21. 2021Improving Robustness using Generated Datacited by 234
  22. 2018Adversarial Logit Pairingcited by 224
  23. 2020Understanding and Improving Fast Adversarial Trainingcited by 214
  24. 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 204
  25. 2020Bag of Tricks for Adversarial Trainingcited by 200
  26. 2020Geometry-aware Instance-reweighted Adversarial Trainingcited by 198
  27. 2018Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networkscited by 195
  28. 2019Adversarial Training and Robustness for Multiple Perturbationscited by 183
  29. 2018Disentangling Adversarial Robustness and Generalizationcited by 172
  30. 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 172
  31. 2019Are Labels Required for Improving Adversarial Robustness?cited by 163
  32. 2020Understanding and Mitigating the Tradeoff Between Robustness and Accuracycited by 149
  33. 2018Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Modelscited by 135
  34. 2018Decoupling Direction and Norm for Efficient Gradient-Based L2 Adversarial Attacks and Defensescited by 109
  35. 2020Boosting Adversarial Training with Hypersphere Embeddingcited by 97
  36. 2020RayS: A Ray Searching Method for Hard-label Adversarial Attackcited by 96
  37. 2019On the Connection Between Adversarial Robustness and Saliency Map Interpretabilitycited by 80
  38. 2020Do Wider Neural Networks Really Help Adversarial Robustness?cited by 79

50 most cited of 80 All 80 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 9; 2023: 45; 2024: 33; 2025: 17; 2026: 7.

111 papers.

Sort:

By area:

  1. 2023On Evaluating Adversarial Robustness of Large Vision-Language Modelscited by 284
  2. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 208
  3. 2023Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Modelscited by 79
  4. 2023How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defensescited by 66
  5. 2023Robust Evaluation of Diffusion-Based Adversarial Purificationcited by 65
  6. 2023Decoupled Kullback-Leibler Divergence Losscited by 58
  7. 2023Improving Fast Adversarial Training with Prior-Guided Knowledgecited by 35
  8. 2022DISCO: Adversarial Defense with Local Implicit Functionscited by 34
  9. 2023Randomized Adversarial Training via Taylor Expansioncited by 28
  10. 2023Improving Generalization of Adversarial Training via Robust Critical Fine-Tuningcited by 23
  11. 2023Enhancing Adversarial Robustness via Score-Based Optimizationcited by 19
  12. 2023Improving the Accuracy-Robustness Trade-Off of Classifiers via Adaptive Smoothingcited by 18
38 more
  1. 2023MimicDiffusion: Purifying Adversarial Perturbation via Mimicking Clean Diffusion Modelcited by 18
  2. 2023Wasserstein distributional robustness of neural networkscited by 16
  3. 2024MixedNUTS: Training-Free Accuracy-Robustness Balance via Nonlinearly Mixed Classifierscited by 16
  4. 2024Adversarial Training on Purification (AToP): Advancing Both Robustness and Generalizationcited by 15
  5. 2023Defenses in Adversarial Machine Learning: A Surveycited by 14
  6. 2022Towards Lightweight Black-Box Attacks against Deep Neural Networkscited by 13
  7. 2023Adversarial Training Should Be Cast as a Non-Zero-Sum Gamecited by 13
  8. 2023Mitigating Accuracy-Robustness Trade-off via Balanced Multi-Teacher Adversarial Distillationcited by 13
  9. 2023Uncovering Adversarial Risks of Test-Time Adaptationcited by 12
  10. 2023MIMIR: Masked Image Modeling for Mutual Information-based Adversarial Robustnesscited by 12
  11. 2024Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Datacited by 12
  12. 2024DAT: Improving Adversarial Robustness via Generative Amplitude Mix-up in Frequency Domaincited by 12
  13. 2023Mitigating Adversarial Vulnerability through Causal Parameter Estimation by Adversarial Double Machine Learningcited by 11
  14. 2025Enhancing Robust Fairness via Confusional Spectral Regularizationcited by 11
  15. 2025Robustness and Cybersecurity in the EU Artificial Intelligence Actcited by 11
  16. 2024Detecting Brittle Decisions for Free: Leveraging Margin Consistency in Deep Robust Classifierscited by 10
  17. 2022A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunitiescited by 9
  18. 2024Shedding More Light on Robust Classifiers under the lens of Energy-based Modelscited by 9
  19. 2024Adversarial Guided Diffusion Models for Adversarial Purificationcited by 8
  20. 2025Diffusion-based Adversarial Purification from the Perspective of the Frequency Domaincited by 8
  21. 2022Understanding the Vulnerability of Skeleton-based Human Activity Recognition via Black-box Attackcited by 7
  22. 2024Calibration Attacks: A Comprehensive Study of Adversarial Attacks on Model Confidencecited by 7
  23. 2024PeerAiD: Improving Adversarial Distillation from a Specialized Peer Tutorcited by 7
  24. 2024Boosting Adversarial Training via Fisher-Rao Norm-based Regularizationcited by 7
  25. 2024Certified Robust Accuracy of Neural Networks Are Bounded due to Bayes Errorscited by 7
  26. 2025RODEO: Robust Outlier Detection via Exposing Adaptive Out-of-Distribution Samplescited by 7
  27. 2025Bypassing Prompt Guards in Production with Controlled-Release Promptingcited by 7
  28. 2022Improving Robust Generalization by Direct PAC-Bayesian Bound Minimizationcited by 6
  29. 2023AROID: Improving Adversarial Robustness Through Online Instance-Wise Data Augmentationcited by 6
  30. 2023On the Clean Generalization and Robust Overfitting in Adversarial Training from Two Theoretical Views: Representation Complexity and Training Dynamicscited by 5
  31. 2023Splitting the Difference on Adversarial Trainingcited by 5
  32. 2023Towards Robust and Accurate Visual Promptingcited by 5
  33. 2023Adversarial Robust Memory-Based Continual Learnercited by 5
  34. 2024SpecFormer: Guarding Vision Transformer Robustness via Maximum Singular Value Penalizationcited by 5
  35. 2024Dynamic Guidance Adversarial Distillation with Enhanced Teacher Knowledgecited by 5
  36. 2023Revisiting the Trade-off between Accuracy and Robustness via Weight Distribution of Filterscited by 4
  37. 2023Enhancing Adversarial Training via Reweighting Optimization Trajectorycited by 4
  38. 2023Improving Fast Minimum-Norm Attacks with Hyperparameter Optimizationcited by 4

50 most cited of 111 All 111 (JSON)