# Theoretical evidence for adversarial robustness through randomization

arXiv: https://arxiv.org/abs/1902.01148 · HTML: https://mlsanity.com/paper/1902.01148/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Feb 2019.

## Built on by (most cited first, 30 of 33)

- [How Does Information Bottleneck Help Deep Learning?](https://mlsanity.com/paper/2305.18887/) (2023)
- [A Dynamical System Perspective for Lipschitz Neural Networks](https://mlsanity.com/paper/2110.12690/) (2021)
- [PointGuard: Provably Robust 3D Point Cloud Classification](https://mlsanity.com/paper/2103.03046/) (2021)
- [Provably Efficient Black-Box Action Poisoning Attacks Against Reinforcement Learning](https://mlsanity.com/paper/2110.04471/) (2021)
- [R-LPIPS: An Adversarially Robust Perceptual Similarity Metric](https://mlsanity.com/paper/2307.15157/) (2023)
- [Noisy Feature Mixup](https://mlsanity.com/paper/2110.02180/) (2021)
- [Mixed Nash Equilibria in the Adversarial Examples Game](https://mlsanity.com/paper/2102.06905/) (2021)
- [On the Effectiveness of Small Input Noise for Defending Against Query-based Black-Box Attacks](https://mlsanity.com/paper/2101.04829/) (2021)
- [Enhancing Quantum Adversarial Robustness by Randomized Encodings](https://mlsanity.com/paper/2212.02531/) (2022)
- [LipSim: A Provably Robust Perceptual Similarity Metric](https://mlsanity.com/paper/2310.18274/) (2023)
- [A Simple and Yet Fairly Effective Defense for Graph Neural Networks](https://mlsanity.com/paper/2402.13987/) (2024)
- [On the robustness of randomized classifiers to adversarial examples](https://mlsanity.com/paper/2102.10875/) (2021)
- [Adversarial Robustness by Design through Analog Computing and Synthetic Gradients](https://mlsanity.com/paper/2101.02115/) (2021)
- [PerturbScore: Connecting Discrete and Continuous Perturbations in NLP](https://mlsanity.com/paper/2310.08889/) (2023)
- [Learning distinct features helps, provably](https://mlsanity.com/paper/2106.06012/) (2021)
- [Pre-trained Encoders in Self-Supervised Learning Improve Secure and Privacy-preserving Supervised Learning](https://mlsanity.com/paper/2212.03334/) (2022)
- [On the Role of Randomization in Adversarially Robust Classification](https://mlsanity.com/paper/2302.07221/) (2023)
- [Exploring Adversarial Robustness of Deep State Space Models](https://mlsanity.com/paper/2406.05532/) (2024)
- [Causal Information Bottleneck Boosts Adversarial Robustness of Deep Neural Network](https://mlsanity.com/paper/2210.14229/) (2022)
- [A Survey of Neural Network Robustness Assessment in Image Recognition](https://mlsanity.com/paper/2404.08285/) (2024)
- [ROPUST: Improving Robustness through Fine-tuning with Photonic Processors and Synthetic Gradients](https://mlsanity.com/paper/2108.04217/) (2021)
- [Rethinking Feature Uncertainty in Stochastic Neural Networks for Adversarial Robustness](https://mlsanity.com/paper/2201.00148/) (2022)
- [Towards Evading the Limits of Randomized Smoothing: A Theoretical Analysis](https://mlsanity.com/paper/2206.01715/) (2022)
- [Metric-Fair Classifier Derandomization](https://mlsanity.com/paper/2206.07826/) (2022)
- [Towards Better Certified Segmentation via Diffusion Models](https://mlsanity.com/paper/2306.09949/) (2023)
- [How Sampling Impacts the Robustness of Stochastic Neural Networks](https://mlsanity.com/paper/2204.10839/) (2022)
- [Unifying Gradients to Improve Real-world Robustness for Deep Networks](https://mlsanity.com/paper/2208.06228/) (2022)
- [Robustifying ℓ_∞ Adversarial Training to the Union of Perturbation Models](https://mlsanity.com/paper/2105.14710/) (2021)
- [Improving the Behaviour of Vision Transformers with Token-consistent Stochastic Layers](https://mlsanity.com/paper/2112.15111/) (2021)
- [Towards Optimal Randomized Strategies in Adversarial Example Game](https://mlsanity.com/paper/2306.16738/) (2023)

## Abstract (arXiv)

This paper investigates the theory of robustness against adversarial attacks. It focuses on the family of randomization techniques that consist in injecting noise in the network at inference time. These techniques have proven effective in many contexts, but lack theoretical arguments. We close this gap by presenting a theoretical analysis of these approaches, hence explaining why they perform well in practice. More precisely, we make two new contributions. The first one relates the randomization rate to robustness to adversarial attacks. This result applies for the general family of exponential distributions, and thus extends and unifies the previous approaches. The second contribution consists in devising a new upper bound on the adversarial generalization gap of randomized neural networks. We support our theoretical claims with a set of experiments.