Adversarially Robust Generalization Just Requires More Unlabeled Data
Not on the map. Why?
Abstract (arXiv)
Neural network robustness has recently been highlighted by the existence of adversarial examples. Many previous works show that the learned networks do not perform well on perturbed test data, and significantly more labeled data is required to achieve adversarially robust generalization. In this paper, we theoretically and empirically show that with just more unlabeled data, we can learn a model with better adversarially robust generalization. The key insight of our results is based on a risk decomposition theorem, in which the expected robust risk is separated into two parts: the stability part which measures the prediction stability in the presence of perturbations, and the accuracy part which evaluates the standard classification accuracy. As the stability part does not depend on any label information, we can optimize this part using unlabeled data. We further prove that for a specific Gaussian mixture problem, adversarially robust generalization can be almost as easy as the standard generalization in supervised learning if a sufficiently large amount of unlabeled data is provided. Inspired by the theoretical findings, we further show that a practical adversarial training algorithm that leverages unlabeled data can improve adversarial robust generalization on MNIST and Cifar-10.
Key facts. Cited by 76 papers in the map (data through 14 Oct 2026). First posted on arXiv 3 Jun 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 23; 2022: 26; 2023: 12; 2024: 7; 2025: 5; 2026: 3.
76 papers.
Sort:
By area:
- 2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 279
- 2021Improving Robustness using Generated Datacited by 234
- 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 210
- 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 208
- 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 204
- 2021Data Augmentation Can Improve Robustnesscited by 194
- 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Modelscited by 168
- 2021Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?cited by 113
- 2021Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferabilitycited by 84
- 2021When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?cited by 77
- 2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 74
- 2021Exploring Memorization in Adversarial Trainingcited by 55
38 more
- 2022Understanding Robust Overfitting of Adversarial Training and Beyondcited by 51
- 2021Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Themcited by 48
- 2021Distributionally Robust Learningcited by 44
- 2022A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologiescited by 44
- 2022Stability Analysis and Generalization Bounds of Adversarial Trainingcited by 30
- 2022Robust Weight Perturbation for Adversarial Trainingcited by 25
- 2022Denoising Masked AutoEncoders Help Robust Classificationcited by 21
- 2022Decoupled Adversarial Contrastive Learning for Self-supervised Adversarial Robustnesscited by 20
- 2022Adversarial Rademacher Complexity of Deep Neural Networkscited by 18
- 2022Do Perceptually Aligned Gradients Imply Adversarial Robustness?cited by 15
- 2023Hyperbolic Contrastive Learningcited by 15
- 2022Why adversarial training can hurt robust accuracycited by 14
- 2022A Characterization of Semi-Supervised Adversarially-Robust PAC Learnabilitycited by 13
- 2021Learning Diverse-Structured Networks for Adversarial Robustnesscited by 12
- 2021Understanding Generalization in Adversarial Training via the Bias-Variance Decompositioncited by 10
- 2021Improving Model Robustness with Latent Distribution Locally and Globallycited by 9
- 2021MixACM: Mixup-Based Robustness Transfer via Distillation of Activated Channel Mapscited by 9
- 2022MaxMatch: Semi-Supervised Learning with Worst-Case Consistencycited by 9
- 2022A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunitiescited by 9
- 2022GeoECG: Data Augmentation via Wasserstein Geodesic Perturbation for Robust Electrocardiogram Predictioncited by 8
- 2022Nash Equilibria and Pitfalls of Adversarial Training in Adversarial Robustness Gamescited by 8
- 2021Domain Invariant Adversarial Learningcited by 7
- 2022Improving Adversarial Robustness with Self-Paced Hard-Class Pair Reweightingcited by 6
- 2023Sup-Norm Convergence of Deep Neural Network Estimator for Nonparametric Regression by Adversarial Trainingcited by 6
- 2024Adversarial Prompt Distillation for Vision-Language Modelscited by 6
- 2023Splitting the Difference on Adversarial Trainingcited by 5
- 2021Multiplicative Reweighting for Robust Neural Network Optimizationcited by 4
- 2021Explaining Adversarial Vulnerability with a Data Sparsity Hypothesiscited by 4
- 2021Simple Post-Training Robustness Using Test Time Augmentations and Random Forestcited by 4
- 2022Unlabeled Data Help: Minimax Analysis and Adversarial Robustnesscited by 4
- 2022Alleviating Robust Overfitting of Adversarial Training With Consistency Regularizationcited by 4
- 2022Why Adversarial Training of ReLU Networks Is Difficult?cited by 4
- 2022Queried Unlabeled Data Improves and Robustifies Class-Incremental Learningcited by 4
- 2022Adaptive Self-Training for Object Detectioncited by 4
- 2024On the Geometry of Regularization in Adversarial Training: High-Dimensional Asymptotics and Generalization Boundscited by 4
- 2021Bridged Adversarial Trainingcited by 3
- 2021Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustnesscited by 3
- 2022The robust way to stack and bag: the local Lipschitz waycited by 3
50 most cited of 76 All 76 (JSON)
- 2026Homogenization of ℓ2-Adversarial Training in High-Dimensions: Exact Dynamics under Stochastic Gradient Descent30 Jun 2026
- 2026Robust Alignment: Harmonizing Clean Accuracy and Adversarial Robustness in Adversarial Training29 Apr 2026
- 2026Does Order Matter : Connecting The Law of Robustness to Robust Generalization24 Feb 2026
- 2025On the Existence of Consistent Adversarial Attacks in High-Dimensional Linear Classification14 Jun 2025
- 2025Understanding Adversarial Training with Energy-based Models28 May 2025
- 2025Provable Robust Overfitting Mitigation in Wasserstein Distributionally Robust Optimization6 Mar 2025
- 2025Boosting Adversarial Robustness and Generalization with Dictionary Structure2 Feb 2025
- 2025Efficient Semi-Supervised Adversarial Training via Latent Clustering-Based Data Reduction15 Jan 2025
- 2024Adversarial Prompt Distillation for Vision-Language Models22 Nov 2024
- 2024On the Geometry of Regularization in Adversarial Training: High-Dimensional Asymptotics and Generalization Bounds21 Oct 2024
- 2024Revisiting Semi-supervised Adversarial Robustness via Noise-aware Online Robust Distillation19 Sep 2024
- 2024Adversaries With Incentives: A Strategic Alternative to Adversarial Robustness17 Jun 2024
38 more
- 2024Bridging the Gap: Rademacher Complexity in Robust and Standard Generalization8 Jun 2024
- 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models24 May 2024
- 2024The Effectiveness of Random Forgetting for Robust Generalization18 Feb 2024
- 2023Splitting the Difference on Adversarial Training3 Oct 2023
- 2023Understanding Robust Overfitting from the Feature Generalization Perspective1 Oct 2023
- 2023Out-Of-Domain Unlabeled Data Improves Generalization29 Sep 2023
- 2023DAD++: Improved Data-free Test Time Adversarial Defense10 Sep 2023
- 2023Enhancing Adversarial Robustness in Low-Label Regime via Adaptively Weighted Regularization and Knowledge Distillation8 Aug 2023
- 2023Vulnerability-Aware Instance Reweighting For Adversarial Training14 Jul 2023
- 2023Sup-Norm Convergence of Deep Neural Network Estimator for Nonparametric Regression by Adversarial Training8 Jul 2023
- 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applications14 Apr 2023
- 2023Better Diffusion Models Further Improve Adversarial Training9 Feb 2023
- 2023Rethinking Robust Contrastive Learning from the Adversarial Perspective5 Feb 2023
- 2023Interpolation for Robust Learning: Data Augmentation on Wasserstein Geodesics4 Feb 2023
- 2023Hyperbolic Contrastive Learning2 Feb 2023
- 2022A Theoretical Study of The Effects of Adversarial Attacks on Sparse Regression21 Dec 2022
- 2022Alternating Objectives Generates Stronger PGD-Based Adversarial Attacks15 Dec 2022
- 2022Adaptive Self-Training for Object Detection7 Dec 2022
- 2022Adversarial Rademacher Complexity of Deep Neural Networks27 Nov 2022
- 2022Improving Adversarial Robustness with Self-Paced Hard-Class Pair Reweighting26 Oct 2022
- 2022Nash Equilibria and Pitfalls of Adversarial Training in Adversarial Robustness Games23 Oct 2022
- 2022A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities17 Oct 2022
- 2022Denoising Masked AutoEncoders Help Robust Classification10 Oct 2022
- 2022Strength-Adaptive Adversarial Training4 Oct 2022
- 2022Stability Analysis and Generalization Bounds of Adversarial Training3 Oct 2022
- 2022MaxMatch: Semi-Supervised Learning with Worst-Case Consistency26 Sep 2022
- 2022A Novel Plug-and-Play Approach for Adversarially Robust Generalization19 Aug 2022
- 2022Two Heads are Better than One: Robust Learning Meets Multi-branch Models17 Aug 2022
- 2022GeoECG: Data Augmentation via Wasserstein Geodesic Perturbation for Robust Electrocardiogram Prediction2 Aug 2022
- 2022Do Perceptually Aligned Gradients Imply Adversarial Robustness?22 Jul 2022
- 2022Decoupled Adversarial Contrastive Learning for Self-supervised Adversarial Robustness22 Jul 2022
- 2022Understanding Robust Overfitting of Adversarial Training and Beyond17 Jun 2022
- 2022Queried Unlabeled Data Improves and Robustifies Class-Incremental Learning15 Jun 2022
- 2022The robust way to stack and bag: the local Lipschitz way1 Jun 2022
- 2022Why Adversarial Training of ReLU Networks Is Difficult?30 May 2022
- 2022Robust Weight Perturbation for Adversarial Training30 May 2022
- 2022Alleviating Robust Overfitting of Adversarial Training With Consistency Regularization24 May 2022
- 2022A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologies26 Mar 2022
50 newest of 76 All 76 (JSON)
- 2024Adversarial Prompt Distillation for Vision-Language Models22 Nov 2024
- 2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 279
- 2021Improving Robustness using Generated Datacited by 234
- 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 210
- 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 204
- 2021Data Augmentation Can Improve Robustnesscited by 194
- 2021Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?cited by 113
- 2021Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferabilitycited by 84
- 2021When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?cited by 77
- 2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 74
- 2021Exploring Memorization in Adversarial Trainingcited by 55
- 2022Understanding Robust Overfitting of Adversarial Training and Beyondcited by 51
- 2021Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Themcited by 48
12 most cited of 72 All 72 (JSON)
- 2026Homogenization of ℓ2-Adversarial Training in High-Dimensions: Exact Dynamics under Stochastic Gradient Descent30 Jun 2026
- 2026Robust Alignment: Harmonizing Clean Accuracy and Adversarial Robustness in Adversarial Training29 Apr 2026
- 2026Does Order Matter : Connecting The Law of Robustness to Robust Generalization24 Feb 2026
- 2025On the Existence of Consistent Adversarial Attacks in High-Dimensional Linear Classification14 Jun 2025
- 2025Understanding Adversarial Training with Energy-based Models28 May 2025
- 2025Provable Robust Overfitting Mitigation in Wasserstein Distributionally Robust Optimization6 Mar 2025
- 2025Boosting Adversarial Robustness and Generalization with Dictionary Structure2 Feb 2025
- 2025Efficient Semi-Supervised Adversarial Training via Latent Clustering-Based Data Reduction15 Jan 2025
- 2024On the Geometry of Regularization in Adversarial Training: High-Dimensional Asymptotics and Generalization Bounds21 Oct 2024
- 2024Revisiting Semi-supervised Adversarial Robustness via Noise-aware Online Robust Distillation19 Sep 2024
- 2024Adversaries With Incentives: A Strategic Alternative to Adversarial Robustness17 Jun 2024
- 2024Bridging the Gap: Rademacher Complexity in Robust and Standard Generalization8 Jun 2024
12 newest of 72 All 72 (JSON)