mlsanityData to 7 Oct 2026

On damage of interpolation to adversarial robustness in regression

arXiv 2601.16070 ↗First posted 22 Jan 2026stat.ML cs.LG math.STcited by 1

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) typically involve a large number of parameters and are trained to achieve zero or near-zero training error. Despite such interpolation, they often exhibit strong generalization performance on unseen data, a phenomenon that has motivated extensive theoretical investigations. Comforting results show that interpolation indeed may not affect the minimax rate of convergence under the squared error loss. In the mean time, DNNs are well known to be highly vulnerable to adversarial perturbations in future inputs. A natural question then arises: Can interpolation also escape from suboptimal performance under a future X-attack? In this paper, we investigate the adversarial robustness of interpolating estimators in a framework of nonparametric regression. A finding is that interpolating estimators must be suboptimal even under a subtle future X-attack, and achieving perfect fitting can substantially damage their robustness. An interesting phenomenon in the high interpolation regime, which we term the curse of simple size, is also revealed and discussed. Numerical experiments support our theoretical findings.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 58 of its 87 references that are in the map. First posted on arXiv 22 Jan 2026.

Builds on

58 of 87 references.

References · 58

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  5. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  6. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  7. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  8. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  9. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  10. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  11. 2017One pixel attack for fooling deep neural networkscited by 579
  12. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 572
38 more
  1. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
  2. 2019Benign Overfitting in Linear Regressioncited by 544
  3. 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 508
  4. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  5. 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 410
  6. 2015Deep Learning with Limited Numerical Precisioncited by 368
  7. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 357
  8. 2014In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learningcited by 339
  9. 2018Certified Defenses against Adversarial Examplescited by 305
  10. 2019Two models of double descent for weak featurescited by 194
  11. 2018Just Interpolate: Kernel "Ridgeless" Regression Can Generalizecited by 190
  12. 2018To understand deep learning we need to understand kernel learningcited by 186
  13. 2020Benign overfitting in ridge regressioncited by 152
  14. 2017The Power of Interpolation: Understanding the Effectiveness of SGD in Modern Over-parametrized Learningcited by 150
  15. 2021A Universal Law of Robustness via Isoperimetrycited by 150
  16. 2019Harmless interpolation of noisy data in regressioncited by 121
  17. 2019How do infinite width bounded norm networks look in function space?cited by 105
  18. 2018Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolatecited by 99
  19. 2018Does data interpolation contradict statistical optimality?cited by 95
  20. 2020Precise Tradeoffs in Adversarial Training for Linear Regressioncited by 76
  21. 2020A law of robustness for two-layers neural networkscited by 54
  22. 2022Dimension free ridge regressioncited by 50
  23. 2019Improving Neural Language Modeling via Adversarial Trainingcited by 49
  24. 2020Convex Geometry and Duality of Over-parameterized Neural Networkscited by 49
  25. 2018Consistency of Interpolation with Laplace Kernels is a High-Dimensional Phenomenoncited by 44
  26. 2020How benign is benign overfitting?cited by 40
  27. 2022The curse of overparametrization in adversarial training: Precise analysis of robust generalization for random features regressioncited by 40
  28. 2021Early-stopped neural networks are consistentcited by 34
  29. 2019Gradient Descent Finds Global Minima for Generalizable Deep Neural Networks of Practical Sizescited by 29
  30. 2023Mind the spikes: Benign overfitting of kernels and neural networks in fixed dimensioncited by 26
  31. 2020Adversarially Robust Estimate and Risk Analysis in Linear Regressioncited by 22
  32. 2023Penalising the biases in norm regularisation enforces sparsitycited by 17
  33. 2023Kernel interpolation generalizes poorlycited by 16
  34. 2021Interpolation can hurt robust generalization even when there is no noisecited by 15
  35. 2023Noisy Interpolation Learning with Shallow Univariate ReLU Networkscited by 11
  36. 2022A Law of Robustness beyond Isoperimetrycited by 9
  37. 2021Ridgeless Interpolation with Shallow ReLU Networks in 1D is Nearest Neighbor Curvature Extrapolation and Provably Generalizes on Lipschitz Functionscited by 8
  38. 2022Adversarially Robust PAC Learnability of Real-Valued Functionscited by 8

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Adversarial Robustness of NTK Neural Networkscited by 0