mlsanityData to 14 Oct 2026

Unlabeled Data Improves Adversarial Robustness

arXiv 1905.13736 ↗First posted 31 May 2019stat.ML cs.CV cs.LGcited by 416

Not on the map. Why?

Abstract (arXiv)

We demonstrate, theoretically and empirically, that adversarial robustness can significantly benefit from semisupervised learning. Theoretically, we revisit the simple Gaussian model of Schmidt et al. that shows a sample complexity gap between standard and robust classification. We prove that unlabeled data bridges this gap: a simple semisupervised learning procedure (self-training) achieves high robust accuracy using the same number of labels required for achieving high standard accuracy. Empirically, we augment CIFAR-10 with 500K unlabeled images sourced from 80 Million Tiny Images and use robust self-training to outperform state-of-the-art robust accuracies by over 5 points in (i) ℓ_∞ robustness against several strong attacks via adversarial training and (ii) certified ℓ2 and ℓ_∞ robustness via randomized smoothing. On SVHN, adding the dataset's own extra training set with the labels removed provides gains of 4 to 10 points, within 1 point of the gain from using the extra labels.

Key facts. Cited by 416 papers in the map (data through 14 Oct 2026). First posted on arXiv 31 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 119; 2022: 113; 2023: 90; 2024: 53; 2025: 33; 2026: 8.

416 papers.

Sort:

By area:

  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  2. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Modelscited by 316
  3. 2021Unsolved Problems in ML Safetycited by 292
  4. 2021Distribution-Free, Risk-Controlling Prediction Setscited by 288
  5. 2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 279
  6. 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 249
  7. 2021Improving Robustness using Generated Datacited by 234
  8. 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 210
  9. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 208
  10. 2021Vision Transformers are Robust Learnerscited by 207
  11. 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 204
  12. 2021Data Augmentation Can Improve Robustnesscited by 194
38 more
  1. 2021The Values Encoded in Machine Learning Researchcited by 172
  2. 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Modelscited by 168
  3. 2022Backdoor Defense via Decoupling the Training Processcited by 153
  4. 2021Adversarial purification with Score-based generative modelscited by 134
  5. 2021Proof Artifact Co-training for Theorem Proving with Language Modelscited by 132
  6. 2021Dataset Inference: Ownership Resolution in Machine Learningcited by 116
  7. 2021Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?cited by 113
  8. 2022Robustness and Accuracy Could Be Reconcilable by (Proper) Definitioncited by 111
  9. 2022Test-Time Adaptation via Conjugate Pseudo-labelscited by 98
  10. 2021Trustworthy AI: A Computational Perspectivecited by 93
  11. 2022Guided Diffusion Model for Adversarial Purificationcited by 92
  12. 2023DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedulecited by 85
  13. 2021Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Bettercited by 79
  14. 2023Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Modelscited by 79
  15. 2021When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?cited by 77
  16. 2021Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimizationcited by 76
  17. 2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 74
  18. 2023How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defensescited by 66
  19. 2022Evaluating the Adversarial Robustness of Adaptive Test-time Defensescited by 64
  20. 2022A Light Recipe to Train Robust Vision Transformerscited by 64
  21. 2021When and How Mixup Improves Calibrationcited by 60
  22. 2021Open-set Label Noise Can Improve Robustness Against Inherent Label Noisecited by 59
  23. 2021Relating Adversarially Robust Generalization to Flat Minimacited by 57
  24. 2021Probabilistic Margins for Instance Reweighting in Adversarial Trainingcited by 57
  25. 2021Reliable Adversarial Distillation with Unreliable Teacherscited by 56
  26. 2021Exploring Memorization in Adversarial Trainingcited by 55
  27. 2022Semi-Supervised and Unsupervised Deep Visual Learning: A Surveycited by 52
  28. 2021Mind the box: l1-APGD for sparse adversarial attacks on image classifierscited by 51
  29. 2022Understanding Robust Overfitting of Adversarial Training and Beyondcited by 51
  30. 2021Fast Minimum-norm Adversarial Attacks through Adaptive Norm Constraintscited by 50
  31. 2021Adversarial Robustness under Long-Tailed Distributioncited by 50
  32. 2021Targeted Attack against Deep Neural Networks via Flipping Limited Weight Bitscited by 49
  33. 2021Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Themcited by 48
  34. 2021Calibration and Consistency of Adversarial Surrogate Lossescited by 47
  35. 2021Distributionally Robust Learningcited by 44
  36. 2022Practical Evaluation of Adversarial Robustness via Adaptive Auto Attackcited by 44
  37. 2022Efficient and Effective Augmentation Strategy for Adversarial Trainingcited by 43
  38. 2023Data Augmentation Alone Can Improve Adversarial Trainingcited by 43

50 most cited of 416 All 416 (JSON)