mlsanityData to 7 Oct 2026

Statistical Guarantees for Regularized Neural Networks

arXiv 2006.00294 ↗First posted 30 May 2020cs.LG cs.NE math.STcited by 25

Not on the map. Why?

Abstract (arXiv)

Neural networks have become standard tools in the analysis of data, but they lack comprehensive mathematical theories. For example, there are very few statistical guarantees for learning neural networks from data, especially for classes of estimators that are used in practice or at least similar to such. In this paper, we develop a general statistical guarantee for estimators that consist of a least-squares term and a regularizer. We then exemplify this guarantee with ℓ1-regularization, showing that the corresponding prediction error increases at most sub-linearly in the number of layers and at most logarithmically in the total number of parameters. Our results establish a mathematical basis for regularized estimation of neural networks, and they deepen our mathematical understanding of neural networks and deep learning more generally.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 6; 2023: 3; 2024: 5; 2025: 4; 2026: 2.

25 papers.

Sort:

By area:

  1. 2021Activation Functions in Artificial Neural Networks: A Systematic Overviewcited by 12
  2. 2024Fast Training of Sinusoidal Neural Fields via Scaling Initializationcited by 11
  3. 2022Deep Learning meets Nonparametric Regression: Are Weight-Decayed DNNs Locally Adaptive?cited by 10
  4. 2022Statistical guarantees for sparse deep learningcited by 9
  5. 2022Non-Asymptotic Guarantees for Robust Statistical Learning under Infinite Variance Assumptioncited by 6
  6. 2021Regularization and Reparameterization Avoid Vanishing Gradients in Sigmoid-Type Networkscited by 5
  7. 2024How many samples are needed to train a deep neural network?cited by 5
  8. 2021Analytic function approximation by path norm regularized deep networkscited by 3
  9. 2021Neural networks with superexpressive activations and integer weightscited by 3
  10. 2024On the estimation rate of Bayesian PINN for inverse problemscited by 3
  11. 2025Residual Neural Terminal Constraint for MPC-based Collision Avoidance in Dynamic Environmentscited by 3
  12. 2021Function approximation by deep neural networks with parameters 0,± frac12, ± 1, 2cited by 2
13 more
  1. 2022A PAC-Bayes oracle inequality for sparse neural networkscited by 2
  2. 2025Regularization can make diffusion models more efficientcited by 2
  3. 2023Adversarial Training with Generated Data in High-Dimensional Regression: An Asymptotic Studycited by 1
  4. 2023Statistical learning by sparse deep neural networkscited by 1
  5. 2024Adversarially robust generalization theory via Jacobian regularization for deep neural networkscited by 1
  6. 2022Nonparametric regression with modified ReLU networkscited by 0
  7. 2022Distribution Estimation of Contaminated Data via DNN-based MoM-GANscited by 0
  8. 2023A statistical perspective on algorithm unrolling models for inverse problemscited by 0
  9. 2024Better Representations via Adversarial Training in Pre-Training: A Theoretical Perspectivecited by 0
  10. 2025How to Enhance Downstream Adversarial Robustness (almost) without Touching the Pre-Trained Foundation Model?cited by 0
  11. 2025Neural Drift Estimation for Ergodic Diffusions: Non-parametric Analysis and Numerical Explorationcited by 0
  12. 2026Regularizing Attention Scores with Bootstrappingcited by 0
  13. 2026Knockoffs-based False Discovery Rate Control and Simplification for Deep Neural Networkscited by 0