# Robustness, Privacy, and Generalization of Adversarial Training

arXiv: https://arxiv.org/abs/2012.13573 · HTML: https://mlsanity.com/paper/2012.13573/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Dec 2020.

## Built on by (most cited first, 8 of 8)

- [Gradient Masking and the Underestimated Robustness Threats of Differential Privacy in Deep Learning](https://mlsanity.com/paper/2105.07985/) (2021)
- [Fast Propagation is Better: Accelerating Single-Step Adversarial Training via Sampling Subnetworks](https://mlsanity.com/paper/2310.15444/) (2023)
- [Probing the Transition to Dataset-Level Privacy in ML Models Using an Output-Specific and Data-Resolved Privacy Profile](https://mlsanity.com/paper/2306.15790/) (2023)
- [OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge Collaborative AutoML System](https://mlsanity.com/paper/2303.00501/) (2023)
- [Combining Machine Learning Defenses without Conflicts](https://mlsanity.com/paper/2411.09776/) (2024)
- [Black-Box Generalization: Stability of Zeroth-Order Learning](https://mlsanity.com/paper/2202.06880/) (2022)
- [Exploring High-Order Structure for Robust Graph Structure Learning](https://mlsanity.com/paper/2203.11492/) (2022)
- [The Double-Edged Nature of the Rashomon Set for Trustworthy Machine Learning](https://mlsanity.com/paper/2511.21799/) (2025)

## Abstract (arXiv)

Adversarial training can considerably robustify deep neural networks to resist adversarial attacks. However, some works suggested that adversarial training might comprise the privacy-preserving and generalization abilities. This paper establishes and quantifies the privacy-robustness trade-off and generalization-robustness trade-off in adversarial training from both theoretical and empirical aspects. We first define a notion, it robustified intensity to measure the robustness of an adversarial training algorithm. This measure can be approximate empirically by an asymptotically consistent empirical estimator, it empirical robustified intensity. Based on the robustified intensity, we prove that (1) adversarial training is (varepsilon, δ)-differentially private, where the magnitude of the differential privacy has a positive correlation with the robustified intensity; and (2) the generalization error of adversarial training can be upper bounded by an mathcal O(√log N/N) on-average bound and an mathcal O(1/√N) high-probability bound, both of which have positive correlations with the robustified intensity. Additionally, our generalization bounds do not explicitly rely on the parameter size which would be prohibitively large in deep learning. Systematic experiments on standard datasets, CIFAR-10 and CIFAR-100, are in full agreement with our theories. The source code package is available at urlhttps://github.com/fshp971/RPG.