# Disentangling Adversarial Robustness and Generalization

arXiv: https://arxiv.org/abs/1812.00740 · HTML: https://mlsanity.com/paper/1812.00740/

## Key facts

- Cited by 172 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2018.

## Built on by (most cited first, 30 of 172)

- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [Recent Advances in Adversarial Training for Adversarial Robustness](https://mlsanity.com/paper/2102.01356/) (2021)
- [Better Diffusion Models Further Improve Adversarial Training](https://mlsanity.com/paper/2302.04638/) (2023)
- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [Adversarial Examples Make Strong Poisons](https://mlsanity.com/paper/2106.10807/) (2021)
- [Understanding Zero-Shot Adversarial Robustness for Large-Scale Models](https://mlsanity.com/paper/2212.07016/) (2022)
- [Robustness and Accuracy Could Be Reconcilable by (Proper) Definition](https://mlsanity.com/paper/2202.10103/) (2022)
- [On the effectiveness of adversarial training against common corruptions](https://mlsanity.com/paper/2103.02325/) (2021)
- [Relating Adversarially Robust Generalization to Flat Minima](https://mlsanity.com/paper/2104.04448/) (2021)
- [How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness?](https://mlsanity.com/paper/2112.11668/) (2021)
- [Online Adversarial Purification based on Self-Supervision](https://mlsanity.com/paper/2101.09387/) (2021)
- [Aug-NeRF: Training Stronger Neural Radiance Fields with Triple-Level Physically-Grounded Augmentations](https://mlsanity.com/paper/2207.01164/) (2022)
- [Noisy Recurrent Neural Networks](https://mlsanity.com/paper/2102.04877/) (2021)
- [Uncertainty-guided Model Generalization to Unseen Domains](https://mlsanity.com/paper/2103.07531/) (2021)
- [The Dimpled Manifold Model of Adversarial Examples in Machine Learning](https://mlsanity.com/paper/2106.10151/) (2021)
- [Generalization of Neural Combinatorial Solvers Through the Lens of Adversarial Robustness](https://mlsanity.com/paper/2110.10942/) (2021)
- [DiffProtect: Generate Adversarial Examples with Diffusion Models for Facial Privacy Protection](https://mlsanity.com/paper/2305.13625/) (2023)
- [A review on data-driven constitutive laws for solids](https://mlsanity.com/paper/2405.03658/) (2024)
- [A Little Robustness Goes a Long Way: Leveraging Robust Features for Targeted Transfer Attacks](https://mlsanity.com/paper/2106.02105/) (2021)
- [Attacking Adversarial Attacks as A Defense](https://mlsanity.com/paper/2106.04938/) (2021)
- [Why Does Little Robustness Help? A Further Step Towards Understanding Adversarial Transferability](https://mlsanity.com/paper/2307.07873/) (2023)
- [BASAR:Black-box Attack on Skeletal Action Recognition](https://mlsanity.com/paper/2103.05266/) (2021)
- [Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective](https://mlsanity.com/paper/2103.00397/) (2021)
- [ADBM: Adversarial diffusion bridge model for reliable adversarial purification](https://mlsanity.com/paper/2408.00315/) (2024)
- [Adversarial Robustness against Multiple and Single lp-Threat Models via Quick Fine-Tuning of Robust Classifiers](https://mlsanity.com/paper/2105.12508/) (2021)
- [Uncertainty-Aware Reliable Text Classification](https://mlsanity.com/paper/2107.07114/) (2021)
- [Exploring the Relationship between Architecture and Adversarially Robust Generalization](https://mlsanity.com/paper/2209.14105/) (2022)
- [TAPT: Test-Time Adversarial Prompt Tuning for Robust Inference in Vision-Language Models](https://mlsanity.com/paper/2411.13136/) (2024)
- [Adversarial Bayesian Augmentation for Single-Source Domain Generalization](https://mlsanity.com/paper/2307.09520/) (2023)
- [Adversarial Training is Not Ready for Robot Learning](https://mlsanity.com/paper/2103.08187/) (2021)

## Abstract (arXiv)

Obtaining deep networks that are robust against adversarial examples and generalize well is an open problem. A recent hypothesis even states that both robust and accurate models are impossible, i.e., adversarial robustness and generalization are conflicting goals. In an effort to clarify the relationship between robustness and generalization, we assume an underlying, low-dimensional data manifold and show that: 1. regular adversarial examples leave the manifold; 2. adversarial examples constrained to the manifold, i.e., on-manifold adversarial examples, exist; 3. on-manifold adversarial examples are generalization errors, and on-manifold adversarial training boosts generalization; 4. regular robustness and generalization are not necessarily contradicting goals. These assumptions imply that both robust and accurate models are possible. However, different models (architectures, training strategies etc.) can exhibit different robustness and generalization characteristics. To confirm our claims, we present extensive experiments on synthetic data (with known manifold) as well as on EMNIST, Fashion-MNIST and CelebA.