# Precise Statistical Analysis of Classification Accuracies for Adversarial Training

arXiv: https://arxiv.org/abs/2010.11213 · HTML: https://mlsanity.com/paper/2010.11213/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2020.

## Built on by (most cited first, 30 of 50)

- [CUDA: Convolution-based Unlearnable Datasets](https://mlsanity.com/paper/2303.04278/) (2023)
- [Overparameterized Linear Regression under Adversarial Attacks](https://mlsanity.com/paper/2204.06274/) (2022)
- [H-Consistency Guarantees for Regression](https://mlsanity.com/paper/2403.19480/) (2024)
- [Regularization properties of adversarially-trained linear regression](https://mlsanity.com/paper/2310.10807/) (2023)
- [Unlearnable 3D Point Clouds: Class-wise Transformation Is All You Need](https://mlsanity.com/paper/2410.03644/) (2024)
- [Interpolation can hurt robust generalization even when there is no noise](https://mlsanity.com/paper/2108.02883/) (2021)
- [Implicit Optimization Bias of Next-Token Prediction in Linear Models](https://mlsanity.com/paper/2402.18551/) (2024)
- [Asymptotics of Non-Convex Generalized Linear Models in High-Dimensions: A proof of the replica formula](https://mlsanity.com/paper/2502.20003/) (2025)
- [Beyond the Universal Law of Robustness: Sharper Laws for Random Features and Neural Tangent Kernels](https://mlsanity.com/paper/2302.01629/) (2023)
- [Fundamental Tradeoffs in Distributionally Adversarial Training](https://mlsanity.com/paper/2101.06309/) (2021)
- [Can Adversarial Training Be Manipulated By Non-Robust Features?](https://mlsanity.com/paper/2201.13329/) (2022)
- [Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data](https://mlsanity.com/paper/2410.08503/) (2024)
- [Benign Overfitting in Adversarially Robust Linear Classification](https://mlsanity.com/paper/2112.15250/) (2021)
- [Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition](https://mlsanity.com/paper/2103.09947/) (2021)
- [Sharp global convergence guarantees for iterative nonconvex optimization: A Gaussian process perspective](https://mlsanity.com/paper/2109.09859/) (2021)
- [Adversarial robustness for latent models: Revisiting the robust-standard accuracies tradeoff](https://mlsanity.com/paper/2110.11950/) (2021)
- [Regularized Linear Regression for Binary Classification](https://mlsanity.com/paper/2311.02270/) (2023)
- [A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offs](https://mlsanity.com/paper/2402.05674/) (2024)
- [Universality of max-margin classifiers](https://mlsanity.com/paper/2310.00176/) (2023)
- [Efficient Optimization Algorithms for Linear Adversarial Training](https://mlsanity.com/paper/2410.12677/) (2024)
- [Bridging the Gap Between Adversarial Robustness and Optimization Bias](https://mlsanity.com/paper/2102.08868/) (2021)
- [A New Central Limit Theorem for the Augmented IPW Estimator: Variance Inflation, Cross-Fit Covariance and Beyond](https://mlsanity.com/paper/2205.10198/) (2022)
- [On the Clean Generalization and Robust Overfitting in Adversarial Training from Two Theoretical Views: Representation Complexity and Training Dynamics](https://mlsanity.com/paper/2306.01271/) (2023)
- [Robust Feature Learning for Multi-Index Models in High Dimensions](https://mlsanity.com/paper/2410.16449/) (2024)
- [Unlabeled Data Help: Minimax Analysis and Adversarial Robustness](https://mlsanity.com/paper/2202.06996/) (2022)
- [Robust Linear Regression: Gradient-descent, Early-stopping, and Beyond](https://mlsanity.com/paper/2301.13486/) (2023)
- [A Novel Gaussian Min-Max Theorem and its Applications](https://mlsanity.com/paper/2402.07356/) (2024)
- [The Breakdown of Gaussian Universality in Classification of High-dimensional Linear Factor Mixtures](https://mlsanity.com/paper/2410.05609/) (2024)
- [On the Geometry of Regularization in Adversarial Training: High-Dimensional Asymptotics and Generalization Bounds](https://mlsanity.com/paper/2410.16073/) (2024)
- [Margin-based sampling in high dimensions: When being active is less efficient than staying passive](https://mlsanity.com/paper/2212.00772/) (2022)

## Abstract (arXiv)

Despite the wide empirical success of modern machine learning algorithms and models in a multitude of applications, they are known to be highly susceptible to seemingly small indiscernible perturbations to the input data known as adversarial attacks. A variety of recent adversarial training procedures have been proposed to remedy this issue. Despite the success of such procedures at increasing accuracy on adversarially perturbed inputs or robust accuracy, these techniques often reduce accuracy on natural unperturbed inputs or standard accuracy. Complicating matters further, the effect and trend of adversarial training procedures on standard and robust accuracy is rather counter intuitive and radically dependent on a variety of factors including the perceived form of the perturbation during training, size/quality of data, model overparameterization, etc. In this paper we focus on binary classification problems where the data is generated according to the mixture of two Gaussians with general anisotropic covariance matrices and derive a precise characterization of the standard and robust accuracy for a class of minimax adversarially trained models. We consider a general norm-based adversarial model, where the adversary can add perturbations of bounded ℓp norm to each input data, for an arbitrary p≥ 1. Our comprehensive analysis allows us to theoretically explain several intriguing empirical phenomena and provide a precise understanding of the role of different problem parameters on standard and robust accuracies.