mlsanityData to 7 Oct 2026

PAC-Bayesian Spectrally-Normalized Bounds for Adversarially Robust Generalization

arXiv 2310.06182 ↗First posted 9 Oct 2023cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) are vulnerable to adversarial attacks. It is found empirically that adversarially robust generalization is crucial in establishing defense algorithms against adversarial attacks. Therefore, it is interesting to study the theoretical guarantee of robust generalization. This paper focuses on norm-based complexity, based on a PAC-Bayes approach (Neyshabur et al., 2017). The main challenge lies in extending the key ingredient, which is a weight perturbation bound in standard settings, to the robust settings. Existing attempts heavily rely on additional strong assumptions, leading to loose bounds. In this paper, we address this issue and provide a spectrally-normalized robust generalization bound for DNNs. Compared to existing bounds, our bound offers two significant advantages: Firstly, it does not depend on additional assumptions. Secondly, it is considerably tighter, aligning with the bounds of standard generalization. Therefore, our result provides a different perspective on understanding robust generalization: The mismatch terms between standard and robust generalization bounds shown in previous studies do not contribute to the poor robust generalization. Instead, these disparities solely due to mathematical issues. Finally, we extend the main result to adversarial robustness against general non-ℓp attacks and other neural network architectures.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 29 of its 41 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

29 of 41 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  6. 2016Adversarial examples in the physical worldcited by 1,762
  7. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  8. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  9. 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
  10. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  11. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  12. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
17 more
  1. 2017Exploring Generalization in Deep Learningcited by 595
  2. 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
  3. 2020Overfitting in adversarially robust deep learningcited by 504
  4. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 309
  5. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  6. 2017Size-Independent Sample Complexity of Neural Networkscited by 289
  7. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 275
  8. 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 201
  9. 2018Rademacher Complexity for Adversarially Robust Generalizationcited by 132
  10. 2018Generalizable Adversarial Training via Spectral Normalizationcited by 68
  11. 2020Adversarial Learning Guarantees for Linear Hypotheses and Neural Networkscited by 47
  12. 2020Dual Manifold Adversarial Robustness: Defense against Lp and non-Lp Adversarial Attackscited by 45
  13. 2022Stability Analysis and Generalization Bounds of Adversarial Trainingcited by 30
  14. 2022Adversarial Rademacher Complexity of Deep Neural Networkscited by 18
  15. 2022What is a Good Metric to Study Generalization of Minimax Learners?cited by 16
  16. 2021A PAC-Bayes Analysis of Adversarial Robustnesscited by 13
  17. 2022Understanding Adversarial Robustness Against On-manifold Adversarial Examplescited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 4; 2026: 3.

9 papers.

  1. 2025Restoring Calibration for Aligned Large Language Models: A Calibration-Aware Fine-Tuning Approachcited by 25
  2. 2025Enhancing Robust Fairness via Confusional Spectral Regularizationcited by 11
  3. 2026Towards A Unified PAC-Bayesian Framework for Norm-based Generalization Boundscited by 4
  4. 2024Uniformly Stable Algorithms for Adversarial Training and Beyondcited by 2
  5. 2024How Learning Dynamics Drive Adversarially Robust Generalization?cited by 0
  6. 2025Generalization Bounds for Equivariant Networks on Markov Datacited by 0
  7. 2025Catastrophic Overfitting, Entropy Gap and Participation Ratio: A Noiseless lp Norm Solution for Fast Adversarial Trainingcited by 0
  8. 2026S2O: Enhancing Adversarial Training with Second-Order Statistics of Weightscited by 0
  9. 2026PAC-Bayesian Adversarially Robust Generalization for Message Passing Graph Neural Networks: A Sensitivity Analysiscited by 0