mlsanityData to 7 Oct 2026

A PAC-Bayes Analysis of Adversarial Robustness

arXiv 2102.11069 ↗First posted 19 Feb 2021cs.LG cs.AI stat.MLcited by 13

Not on the map. Why?

Abstract (arXiv)

We propose the first general PAC-Bayesian generalization bounds for adversarial robustness, that estimate, at test time, how much a model will be invariant to imperceptible perturbations in the input. Instead of deriving a worst-case analysis of the risk of a hypothesis over all the possible perturbations, we leverage the PAC-Bayesian framework to bound the averaged risk on the perturbations for majority votes (over the whole class of hypotheses). Our theoretically founded analysis has the advantage to provide general bounds (i) that are valid for any kind of attacks (i.e., the adversarial attacks), (ii) that are tight thanks to the PAC-Bayesian framework, (iii) that can be directly minimized during the learning phase to obtain a robust model on different attacks at test time.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 30 of its 48 references that are in the map. First posted on arXiv 19 Feb 2021.

Builds on

30 of 48 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  7. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  8. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  9. 2016Adversarial Machine Learning at Scalecited by 934
  10. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  11. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 739
  12. 2019Provably Robust Deep Learning via Adversarially Trained Smoothed Classifierscited by 328
18 more
  1. 2016Safety Verification of Deep Neural Networkscited by 262
  2. 2019Uniform convergence may be unable to explain generalization in deep learningcited by 223
  3. 2018Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networkscited by 193
  4. 2018Rademacher Complexity for Adversarially Robust Generalizationcited by 132
  5. 2019VC Classes are Adversarially Robustly Learnable, but Only Improperlycited by 83
  6. 2017Entropy-SGD optimizes the prior of a PAC-Bayes bound: Generalization properties of Entropy-SGD and data-dependent priorscited by 81
  7. 2018Generalizable Adversarial Training via Spectral Normalizationcited by 68
  8. 2020On the role of data in PAC-Bayes boundscited by 64
  9. 2017Efficient Defenses Against Adversarial Attackscited by 53
  10. 2015Risk Bounds for the Majority Vote: From a PAC-Bayesian Analysis to a Learning Algorithmcited by 50
  11. 2020Second Order PAC-Bayesian Bounds for the Weighted Majority Votecited by 34
  12. 2020Reducing Adversarially Robust Learning to Non-Robust PAC Learningcited by 28
  13. 2018Learning Gaussian Processes by Minimizing PAC-Bayesian Generalization Boundscited by 24
  14. 2014Majority Vote of Diverse Classifiers for Late Fusioncited by 23
  15. 2018Adversarial Risk Bounds via Function Transformationcited by 12
  16. 2018On PAC-Bayesian Bounds for Random Forestscited by 10
  17. 2018Multiview Boosting by Controlling the Diversity and the Accuracy of View-specific Voterscited by 4
  18. 2014PAC-Bayes Analysis of Multi-view Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 2; 2024: 4; 2025: 1; 2026: 3.

13 papers.

Sort:

  1. 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 206
  2. 2023PAC-Bayesian Spectrally-Normalized Bounds for Adversarially Robust Generalizationcited by 9
  3. 2022Improving Robust Generalization by Direct PAC-Bayesian Bound Minimizationcited by 6
  4. 2024On the Computability of Robust PAC Learningcited by 6
  5. 2026Towards A Unified PAC-Bayesian Framework for Norm-based Generalization Boundscited by 4
  6. 2024Tighter Generalisation Bounds via Interpolationcited by 2
  7. 2025Reconcile Certified Robustness and Accuracy for DNN-based Smoothed Majority Vote Classifiercited by 2
  8. 2022When are Local Queries Useful for Robust Learning?cited by 1
  9. 2024Multi-View Majority Vote Learning Algorithms: Direct Minimization of PAC-Bayesian Boundscited by 1
  10. 2026Explanation-Guided Adversarial Training for Robust and Interpretable Modelscited by 1
  11. 2022Adaptive adversarial training method for improving multi-scale GAN based on generalization bound theorycited by 0
  12. 2024Tightening the Evaluation of PAC Bounds Using Formal Verification Resultscited by 0
1 more
  1. 2026PAC-Bayesian Adversarially Robust Generalization for Message Passing Graph Neural Networks: A Sensitivity Analysiscited by 0