# Adversarially Robust Estimate and Risk Analysis in Linear Regression

arXiv: https://arxiv.org/abs/2012.10278 · HTML: https://mlsanity.com/paper/2012.10278/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Dec 2020.

## Built on by (most cited first, 22 of 22)

- [Cross-Entropy Loss Functions: Theoretical Analysis and Applications](https://mlsanity.com/paper/2304.07288/) (2023)
- [Stability Analysis and Generalization Bounds of Adversarial Training](https://mlsanity.com/paper/2210.00960/) (2022)
- [H-Consistency Guarantees for Regression](https://mlsanity.com/paper/2403.19480/) (2024)
- [Adversarial Rademacher Complexity of Deep Neural Networks](https://mlsanity.com/paper/2211.14966/) (2022)
- [A Characterization of Semi-Supervised Adversarially-Robust PAC Learnability](https://mlsanity.com/paper/2202.05420/) (2022)
- [Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition](https://mlsanity.com/paper/2103.09947/) (2021)
- [Adversarial robustness for latent models: Revisiting the robust-standard accuracies tradeoff](https://mlsanity.com/paper/2110.11950/) (2021)
- [Adversarially Robust PAC Learnability of Real-Valued Functions](https://mlsanity.com/paper/2206.12977/) (2022)
- [Unlabeled Data Help: Minimax Analysis and Adversarial Robustness](https://mlsanity.com/paper/2202.06996/) (2022)
- [Robust Linear Regression: Gradient-descent, Early-stopping, and Beyond](https://mlsanity.com/paper/2301.13486/) (2023)
- [Lagrangian Objective Function Leads to Improved Unforeseen Attack Generalization in Adversarial Training](https://mlsanity.com/paper/2103.15385/) (2021)
- [Surprises in adversarially-trained linear regression](https://mlsanity.com/paper/2205.12695/) (2022)
- [Robust Linear Regression: Phase-Transitions and Precise Tradeoffs for General Norms](https://mlsanity.com/paper/2308.00556/) (2023)
- [Non-Asymptotic Bounds for Adversarial Excess Risk under Misspecified Models](https://mlsanity.com/paper/2309.00771/) (2023)
- [Minimax rates of convergence for nonparametric regression under adversarial attacks](https://mlsanity.com/paper/2410.09402/) (2024)
- [Blessings and Curses of Covariate Shifts: Adversarial Learning Dynamics, Directional Convergence, and Equilibria](https://mlsanity.com/paper/2212.02457/) (2022)
- [A Theoretical Study of The Effects of Adversarial Attacks on Sparse Regression](https://mlsanity.com/paper/2212.11209/) (2022)
- [Adversarial Training with Generated Data in High-Dimensional Regression: An Asymptotic Study](https://mlsanity.com/paper/2306.12582/) (2023)
- [PUMA: margin-based data pruning](https://mlsanity.com/paper/2405.06298/) (2024)
- [Adversarial Robustness of Nonparametric Regression](https://mlsanity.com/paper/2505.17356/) (2025)
- [On damage of interpolation to adversarial robustness in regression](https://mlsanity.com/paper/2601.16070/) (2026)
- [Information Hidden in Gradients of Regression with Target Noise](https://mlsanity.com/paper/2601.18546/) (2026)

## Abstract (arXiv)

Adversarially robust learning aims to design algorithms that are robust to small adversarial perturbations on input variables. Beyond the existing studies on the predictive performance to adversarial samples, our goal is to understand statistical properties of adversarially robust estimates and analyze adversarial risk in the setup of linear regression models. By discovering the statistical minimax rate of convergence of adversarially robust estimators, we emphasize the importance of incorporating model information, e.g., sparsity, in adversarially robust learning. Further, we reveal an explicit connection of adversarial and standard estimates, and propose a straightforward two-stage adversarial learning framework, which facilitates to utilize model structure information to improve adversarial robustness. In theory, the consistency of the adversarially robust estimator is proven and its Bahadur representation is also developed for the statistical inference purpose. The proposed estimator converges in a sharp rate under either low-dimensional or sparse scenario. Moreover, our theory confirms two phenomena in adversarially robust learning: adversarial robustness hurts generalization, and unlabeled data help improve the generalization. In the end, we conduct numerical simulations to verify our theory.