# Adversarially Robust Generalization Just Requires More Unlabeled Data

arXiv: https://arxiv.org/abs/1906.00555 · HTML: https://mlsanity.com/paper/1906.00555/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jun 2019.

## Built on by (most cited first, 30 of 76)

- [Recent Advances in Adversarial Training for Adversarial Robustness](https://mlsanity.com/paper/2102.01356/) (2021)
- [Improving Robustness using Generated Data](https://mlsanity.com/paper/2110.09468/) (2021)
- [Cross-Entropy Loss Functions: Theoretical Analysis and Applications](https://mlsanity.com/paper/2304.07288/) (2023)
- [Better Diffusion Models Further Improve Adversarial Training](https://mlsanity.com/paper/2302.04638/) (2023)
- [Fixing Data Augmentation to Improve Adversarial Robustness](https://mlsanity.com/paper/2103.01946/) (2021)
- [Data Augmentation Can Improve Robustness](https://mlsanity.com/paper/2111.05328/) (2021)
- [Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models](https://mlsanity.com/paper/2405.15234/) (2024)
- [Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?](https://mlsanity.com/paper/2104.09425/) (2021)
- [Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferability](https://mlsanity.com/paper/2111.10752/) (2021)
- [When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?](https://mlsanity.com/paper/2111.01124/) (2021)
- [Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks](https://mlsanity.com/paper/2110.03825/) (2021)
- [Exploring Memorization in Adversarial Training](https://mlsanity.com/paper/2106.01606/) (2021)
- [Understanding Robust Overfitting of Adversarial Training and Beyond](https://mlsanity.com/paper/2206.08675/) (2022)
- [Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Them](https://mlsanity.com/paper/2107.11630/) (2021)
- [Distributionally Robust Learning](https://mlsanity.com/paper/2108.08993/) (2021)
- [A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologies](https://mlsanity.com/paper/2203.14046/) (2022)
- [Stability Analysis and Generalization Bounds of Adversarial Training](https://mlsanity.com/paper/2210.00960/) (2022)
- [Robust Weight Perturbation for Adversarial Training](https://mlsanity.com/paper/2205.14826/) (2022)
- [Denoising Masked AutoEncoders Help Robust Classification](https://mlsanity.com/paper/2210.06983/) (2022)
- [Decoupled Adversarial Contrastive Learning for Self-supervised Adversarial Robustness](https://mlsanity.com/paper/2207.10899/) (2022)
- [Adversarial Rademacher Complexity of Deep Neural Networks](https://mlsanity.com/paper/2211.14966/) (2022)
- [Do Perceptually Aligned Gradients Imply Adversarial Robustness?](https://mlsanity.com/paper/2207.11378/) (2022)
- [Hyperbolic Contrastive Learning](https://mlsanity.com/paper/2302.01409/) (2023)
- [Why adversarial training can hurt robust accuracy](https://mlsanity.com/paper/2203.02006/) (2022)
- [A Characterization of Semi-Supervised Adversarially-Robust PAC Learnability](https://mlsanity.com/paper/2202.05420/) (2022)
- [Learning Diverse-Structured Networks for Adversarial Robustness](https://mlsanity.com/paper/2102.01886/) (2021)
- [Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition](https://mlsanity.com/paper/2103.09947/) (2021)
- [Improving Model Robustness with Latent Distribution Locally and Globally](https://mlsanity.com/paper/2107.04401/) (2021)
- [MixACM: Mixup-Based Robustness Transfer via Distillation of Activated Channel Maps](https://mlsanity.com/paper/2111.05073/) (2021)
- [MaxMatch: Semi-Supervised Learning with Worst-Case Consistency](https://mlsanity.com/paper/2209.12611/) (2022)

## Abstract (arXiv)

Neural network robustness has recently been highlighted by the existence of adversarial examples. Many previous works show that the learned networks do not perform well on perturbed test data, and significantly more labeled data is required to achieve adversarially robust generalization. In this paper, we theoretically and empirically show that with just more unlabeled data, we can learn a model with better adversarially robust generalization. The key insight of our results is based on a risk decomposition theorem, in which the expected robust risk is separated into two parts: the stability part which measures the prediction stability in the presence of perturbations, and the accuracy part which evaluates the standard classification accuracy. As the stability part does not depend on any label information, we can optimize this part using unlabeled data. We further prove that for a specific Gaussian mixture problem, adversarially robust generalization can be almost as easy as the standard generalization in supervised learning if a sufficiently large amount of unlabeled data is provided. Inspired by the theoretical findings, we further show that a practical adversarial training algorithm that leverages unlabeled data can improve adversarial robust generalization on MNIST and Cifar-10.