mlsanityData to 7 Oct 2026

Sup-Norm Convergence of Deep Neural Network Estimator for Nonparametric Regression by Adversarial Training

arXiv 2307.04042 ↗First posted 8 Jul 2023stat.ML cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

We show the sup-norm convergence of deep neural network estimators with a novel adversarial training scheme. For the nonparametric regression problem, it has been shown that an estimator using deep neural networks can achieve better performances in the sense of the L2-norm. In contrast, it is difficult for the neural estimator with least-squares to achieve the sup-norm convergence, due to the deep structure of neural network models. In this study, we develop an adversarial training scheme and investigate the sup-norm convergence of deep neural network estimators. First, we find that ordinary adversarial training makes neural estimators inconsistent. Second, we show that a deep neural network estimator achieves the optimal rate in the sup-norm sense by the proposed adversarial training with correction. We extend our adversarial training to general setups of a loss function and a data-generating function. Our experiments support the theoretical findings.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 30 of its 76 references that are in the map. First posted on arXiv 8 Jul 2023.

Builds on

30 of 76 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  6. 2016Adversarial Machine Learning at Scalecited by 934
  7. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  8. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  9. 2017The Expressive Power of Neural Networks: A View from the Widthcited by 318
  10. 2018Adversarially Robust Generalization Requires More Datacited by 318
  11. 2020Deep Network Approximation for Smooth Functionscited by 163
  12. 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 160
18 more
  1. 2017The Space of Transferable Adversarial Examplescited by 156
  2. 2018Rademacher Complexity for Adversarially Robust Generalizationcited by 132
  3. 2021On the Convergence and Robustness of Adversarial Trainingcited by 127
  4. 2019VC Classes are Adversarially Robustly Learnable, but Only Improperlycited by 83
  5. 2020Precise Tradeoffs in Adversarial Training for Linear Regressioncited by 76
  6. 2019Adversarially Robust Generalization Just Requires More Unlabeled Datacited by 75
  7. 2019Deep ReLU network approximation of functions on a manifoldcited by 73
  8. 2018Generalizable Adversarial Training via Spectral Normalizationcited by 68
  9. 2018Deep Neural Networks for Estimation and Inferencecited by 62
  10. 2018Deep Neural Networks Learn Non-Smooth Functions Effectivelycited by 57
  11. 2019Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov spacecited by 46
  12. 2022The curse of overparametrization in adversarial training: Precise analysis of robust generalization for random features regressioncited by 40
  13. 2015Optimal Sup-norm Rates and Uniform Inference on Nonlinear Functionals of Nonparametric IV Regressioncited by 30
  14. 2022Overparameterized Linear Regression under Adversarial Attackscited by 21
  15. 2020Advantage of Deep Neural Networks for Estimating Functions with Singularity on Hypersurfacescited by 17
  16. 2017Frequentist coverage and sup-norm convergence rate in Gaussian process regressioncited by 15
  17. 2020Estimation error analysis of deep learning on the regression problem on the variable exponent Besov spacecited by 13
  18. 2021Fundamental Tradeoffs in Distributionally Adversarial Trainingcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 0; 2026: 3.

6 papers.

  1. 2024META-ANOVA: Screening interactions for interpretable machine learningcited by 2
  2. 2024Minimax rates of convergence for nonparametric regression under adversarial attackscited by 2
  3. 2026Transfer Learning Through Conditional Quantile Matchingcited by 1
  4. 2024An integrated perspective of robustness in regression through the lens of the bias-variance trade-offcited by 0
  5. 2026Mitigating the Curse of Dimensionality in Uniform Convergence of Deep Neural Networks via Smooth Activationscited by 0
  6. 2026SPARK: A General Goodness-of-Fit Assessment via Residual Projectioncited by 0