Robust Linear Regression: Phase-Transitions and Precise Tradeoffs for General Norms
Not on the map. Why?
Abstract (arXiv)
In this paper, we investigate the impact of test-time adversarial attacks on linear regression models and determine the optimal level of robustness that any model can reach while maintaining a given level of standard predictive performance (accuracy). Through quantitative estimates, we uncover fundamental tradeoffs between adversarial robustness and accuracy in different regimes. We obtain a precise characterization which distinguishes between regimes where robustness is achievable without hurting standard accuracy and regimes where a tradeoff might be unavoidable. Our findings are empirically confirmed with simple experiments that represent a variety of settings. This work applies to feature covariance matrices and attack norms of any nature, and extends beyond previous works in this area.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 23 of its 39 references that are in the map. First posted on arXiv 1 Aug 2023.
Builds on
23 of 39 references.
References · 23
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2013Intriguing properties of neural networkscited by 5,114
- 2018Robustness May Be at Odds with Accuracycited by 873
- 2021Explaining Neural Scaling Lawscited by 353
- 2018Adversarially Robust Generalization Requires More Datacited by 318
- 2018Just Interpolate: Kernel "Ridgeless" Regression Can Generalizecited by 190
- 2021A Universal Law of Robustness via Isoperimetrycited by 150
- 2018Are adversarial examples inevitable?cited by 113
- 2021Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regimecited by 95
- 2022More Than a Toy: Random Matrix Models Predict How Real-World Neural Representations Generalizecited by 87
- 2018Adversarial examples from computational constraintscited by 79
- 2020Precise Tradeoffs in Adversarial Training for Linear Regressioncited by 76
11 more
- 2019Convergence of Adversarial Training in Overparametrized Neural Networkscited by 58
- 2020A law of robustness for two-layers neural networkscited by 54
- 2019Lower Bounds on Adversarial Robustness from Optimal Transportcited by 52
- 2020Precise Statistical Analysis of Classification Accuracies for Adversarial Trainingcited by 49
- 2018The Curse of Concentration in Robust Learning: Evasion and Poisoning Attacks from Concentration of Measurecited by 44
- 2022The curse of overparametrization in adversarial training: Precise analysis of robust generalization for random features regressioncited by 40
- 2020Provable tradeoffs in adversarially robust classificationcited by 39
- 2020Adversarially Robust Estimate and Risk Analysis in Linear Regressioncited by 22
- 2021Fundamental tradeoffs between memorization and robustness in random features and neural tangent regimescited by 9
- 2021Adversarial robustness for latent models: Revisiting the robust-standard accuracies tradeoffcited by 8
- 2023Robust Linear Regression: Gradient-descent, Early-stopping, and Beyondcited by 4
Built on by
Citing papers per year: 2025: 1.
1 paper.