# Robustness of Bayesian Neural Networks to Gradient-Based Attacks

arXiv: https://arxiv.org/abs/2002.04359 · HTML: https://mlsanity.com/paper/2002.04359/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2020.

## Built on by (most cited first, 30 of 48)

- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [Making Substitute Models More Bayesian Can Enhance Transferability of Adversarial Examples](https://mlsanity.com/paper/2302.05086/) (2023)
- [Dangers of Bayesian Model Averaging under Covariate Shift](https://mlsanity.com/paper/2106.11905/) (2021)
- [Bayesian Inference with Certifiable Adversarial Robustness](https://mlsanity.com/paper/2102.05289/) (2021)
- [On the Robustness of Bayesian Neural Networks to Adversarial Attacks](https://mlsanity.com/paper/2207.06154/) (2022)
- [Audio-based AI classifiers show no evidence of improved COVID-19 screening over simple symptoms checkers](https://mlsanity.com/paper/2212.08570/) (2022)
- [Robust Bayesian Learning for Reliable Wireless AI: Framework and Applications](https://mlsanity.com/paper/2207.00300/) (2022)
- [A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities](https://mlsanity.com/paper/2210.08906/) (2022)
- [Bayesian Neural Networks Avoid Encoding Complex and Perturbation-Sensitive Concepts](https://mlsanity.com/paper/2302.13095/) (2023)
- [It Is All About Data: A Survey on the Effects of Data on Adversarial Robustness](https://mlsanity.com/paper/2303.09767/) (2023)
- [Certification of Iterative Predictions in Bayesian Neural Networks](https://mlsanity.com/paper/2105.10134/) (2021)
- [BNN-DP: Robustness Certification of Bayesian Neural Networks via Dynamic Programming](https://mlsanity.com/paper/2306.10742/) (2023)
- [Adversarial Robustness Certification for Bayesian Neural Networks](https://mlsanity.com/paper/2306.13614/) (2023)
- [Adversarial Robustness Guarantees for Gaussian Processes](https://mlsanity.com/paper/2104.03180/) (2021)
- [Adversarial for Good? How the Adversarial ML Community's Values Impede Socially Beneficial Uses of Attacks](https://mlsanity.com/paper/2107.10302/) (2021)
- [A Bayesian approach to quantifying uncertainties and improving generalizability in traffic prediction models](https://mlsanity.com/paper/2307.05946/) (2023)
- [Probabilistic Reach-Avoid for Bayesian Neural Networks](https://mlsanity.com/paper/2310.01951/) (2023)
- [Tight Verification of Probabilistic Robustness in Bayesian Neural Networks](https://mlsanity.com/paper/2401.11627/) (2024)
- [Robustness of Bayesian Neural Networks to White-Box Adversarial Attacks](https://mlsanity.com/paper/2111.08591/) (2021)
- [Bayesian Learning with Information Gain Provably Bounds Risk for a Robust Adversarial Defense](https://mlsanity.com/paper/2212.02003/) (2022)
- [The Effect of Prior Lipschitz Continuity on the Adversarial Robustness of Bayesian Neural Networks](https://mlsanity.com/paper/2101.02689/) (2021)
- [Learning Uncertainty with Artificial Neural Networks for Improved Predictive Process Monitoring](https://mlsanity.com/paper/2206.06317/) (2022)
- [On the Robustness of Randomized Ensembles to Adversarial Perturbations](https://mlsanity.com/paper/2302.01375/) (2023)
- [Individual Fairness in Bayesian Neural Networks](https://mlsanity.com/paper/2304.10828/) (2023)
- [Improving Transferability of Adversarial Examples via Bayesian Attacks](https://mlsanity.com/paper/2307.11334/) (2023)
- [Poisoning Bayesian Inference via Data Deletion and Replication](https://mlsanity.com/paper/2503.04480/) (2025)
- [Variational Routing: A Scalable Bayesian Framework for Calibrated Mixture-of-Experts Transformers](https://mlsanity.com/paper/2603.09453/) (2026)
- [Adversarial Examples Detection with Bayesian Neural Network](https://mlsanity.com/paper/2105.08620/) (2021)
- [Certification of Distributional Individual Fairness](https://mlsanity.com/paper/2311.11911/) (2023)
- [Attacking Bayes: On the Adversarial Robustness of Bayesian Neural Networks](https://mlsanity.com/paper/2404.19640/) (2024)

## Abstract (arXiv)

Vulnerability to adversarial attacks is one of the principal hurdles to the adoption of deep learning in safety-critical applications. Despite significant efforts, both practical and theoretical, the problem remains open. In this paper, we analyse the geometry of adversarial attacks in the large-data, overparametrized limit for Bayesian Neural Networks (BNNs). We show that, in the limit, vulnerability to gradient-based attacks arises as a result of degeneracy in the data distribution, i.e., when the data lies on a lower-dimensional submanifold of the ambient space. As a direct consequence, we demonstrate that in the limit BNN posteriors are robust to gradient-based adversarial attacks. Experimental results on the MNIST and Fashion MNIST datasets with BNNs trained with Hamiltonian Monte Carlo and Variational Inference support this line of argument, showing that BNNs can display both high accuracy and robustness to gradient based adversarial attacks.