# Statistical Guarantees for Regularized Neural Networks

arXiv: https://arxiv.org/abs/2006.00294 · HTML: https://mlsanity.com/paper/2006.00294/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2020.

## Built on by (most cited first, 25 of 25)

- [Activation Functions in Artificial Neural Networks: A Systematic Overview](https://mlsanity.com/paper/2101.09957/) (2021)
- [Fast Training of Sinusoidal Neural Fields via Scaling Initialization](https://mlsanity.com/paper/2410.04779/) (2024)
- [Deep Learning meets Nonparametric Regression: Are Weight-Decayed DNNs Locally Adaptive?](https://mlsanity.com/paper/2204.09664/) (2022)
- [Statistical guarantees for sparse deep learning](https://mlsanity.com/paper/2212.05427/) (2022)
- [Non-Asymptotic Guarantees for Robust Statistical Learning under Infinite Variance Assumption](https://mlsanity.com/paper/2201.03182/) (2022)
- [Regularization and Reparameterization Avoid Vanishing Gradients in Sigmoid-Type Networks](https://mlsanity.com/paper/2106.02260/) (2021)
- [How many samples are needed to train a deep neural network?](https://mlsanity.com/paper/2405.16696/) (2024)
- [Analytic function approximation by path norm regularized deep networks](https://mlsanity.com/paper/2104.02095/) (2021)
- [Neural networks with superexpressive activations and integer weights](https://mlsanity.com/paper/2105.09917/) (2021)
- [On the estimation rate of Bayesian PINN for inverse problems](https://mlsanity.com/paper/2406.14808/) (2024)
- [Residual Neural Terminal Constraint for MPC-based Collision Avoidance in Dynamic Environments](https://mlsanity.com/paper/2508.03428/) (2025)
- [Function approximation by deep neural networks with parameters 0,± frac12, ± 1, 2](https://mlsanity.com/paper/2103.08659/) (2021)
- [A PAC-Bayes oracle inequality for sparse neural networks](https://mlsanity.com/paper/2204.12392/) (2022)
- [Regularization can make diffusion models more efficient](https://mlsanity.com/paper/2502.09151/) (2025)
- [Adversarial Training with Generated Data in High-Dimensional Regression: An Asymptotic Study](https://mlsanity.com/paper/2306.12582/) (2023)
- [Statistical learning by sparse deep neural networks](https://mlsanity.com/paper/2311.08845/) (2023)
- [Adversarially robust generalization theory via Jacobian regularization for deep neural networks](https://mlsanity.com/paper/2412.12449/) (2024)
- [Nonparametric regression with modified ReLU networks](https://mlsanity.com/paper/2207.08306/) (2022)
- [Distribution Estimation of Contaminated Data via DNN-based MoM-GANs](https://mlsanity.com/paper/2212.13741/) (2022)
- [A statistical perspective on algorithm unrolling models for inverse problems](https://mlsanity.com/paper/2311.06395/) (2023)
- [Better Representations via Adversarial Training in Pre-Training: A Theoretical Perspective](https://mlsanity.com/paper/2401.15248/) (2024)
- [How to Enhance Downstream Adversarial Robustness (almost) without Touching the Pre-Trained Foundation Model?](https://mlsanity.com/paper/2504.10850/) (2025)
- [Neural Drift Estimation for Ergodic Diffusions: Non-parametric Analysis and Numerical Exploration](https://mlsanity.com/paper/2505.24383/) (2025)
- [Regularizing Attention Scores with Bootstrapping](https://mlsanity.com/paper/2604.01339/) (2026)
- [Knockoffs-based False Discovery Rate Control and Simplification for Deep Neural Networks](https://mlsanity.com/paper/2606.04404/) (2026)

## Abstract (arXiv)

Neural networks have become standard tools in the analysis of data, but they lack comprehensive mathematical theories. For example, there are very few statistical guarantees for learning neural networks from data, especially for classes of estimators that are used in practice or at least similar to such. In this paper, we develop a general statistical guarantee for estimators that consist of a least-squares term and a regularizer. We then exemplify this guarantee with ℓ1-regularization, showing that the corresponding prediction error increases at most sub-linearly in the number of layers and at most logarithmically in the total number of parameters. Our results establish a mathematical basis for regularized estimation of neural networks, and they deepen our mathematical understanding of neural networks and deep learning more generally.