mlsanityData to 7 Oct 2026

A Priori Estimates of the Population Risk for Two-layer Neural Networks

arXiv 1810.06397 ↗First posted 15 Oct 2018stat.ML cs.LG math.STcited by 76

Not on the map. Why?

Abstract (arXiv)

New estimates for the population risk are established for two-layer neural networks. These estimates are nearly optimal in the sense that the error rates scale in the same way as the Monte Carlo error rates. They are equally effective in the over-parametrized regime when the network size is much larger than the size of the dataset. These new estimates are a priori in nature in the sense that the bounds depend only on some norms of the underlying functions to be fitted, not the parameters in the model, in contrast with most existing results which are a posteriori in nature. Using these a priori estimates, we provide a perspective for understanding why two-layer neural networks perform better than the related kernel methods.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 13; 2023: 17; 2024: 13; 2025: 7; 2026: 8.

76 papers.

Sort:

By area:

  1. 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 95
  2. 2022Overview frequency principle/spectral bias in deep learningcited by 68
  3. 2023On the Generalization Properties of Diffusion Modelscited by 67
  4. 2022Deep Nonparametric Estimation of Operators between Infinite Dimensional Spacescited by 45
  5. 2023The Implicit Regularization of Dynamical Stability in Stochastic Gradient Descentcited by 44
  6. 2024Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscapecited by 43
  7. 2021Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neuronscited by 41
  8. 2021A Framework for Machine Learning of Model Error in Dynamical Systemscited by 37
  9. 2021Random feature neural networks learn Black-Scholes type PDEs without curse of dimensionalitycited by 34
  10. 2023Nonparametric regression using over-parameterized shallow ReLU neural networkscited by 24
  11. 2023Theoretical Analysis of Inductive Biases in Deep Convolutional Networkscited by 21
  12. 2023Sampling weights of deep neural networkscited by 20
38 more
  1. 2021Reproducing Activation Function for Deep Learningcited by 19
  2. 2021On Learnability via Gradient Method for Two-Layer ReLU Neural Networks in Teacher-Student Settingcited by 16
  3. 2022Duality for Neural Networks through Reproducing Kernel Banach Spacescited by 15
  4. 2023A Brief Survey on the Approximation Theory for Sequence Modellingcited by 15
  5. 2022Generalization Error Bounds for Deep Neural Networks Trained by SGDcited by 14
  6. 2021An L2 Analysis of Reinforcement Learning in High Dimensions with Kernel and Neural Network Approximationcited by 13
  7. 2023On the Optimal Expressive Power of ReLU DNNs and Its Application in Approximation with Kolmogorov Superposition Theoremcited by 13
  8. 2021Solving PDEs on Unknown Manifolds with Machine Learningcited by 12
  9. 2021Deep Network Approximation in Terms of Intrinsic Parameterscited by 12
  10. 2024Mean-field Analysis on Two-layer Neural Networks from a Kernel Perspectivecited by 12
  11. 2021Stationary Density Estimation of Ito Diffusions Using Deep Learningcited by 9
  12. 2021Optimal learning of high-dimensional classification problems using deep neural networkscited by 9
  13. 2021Dual Training of Energy-Based Models with Overparametrized Shallow Neural Networkscited by 8
  14. 2022Approximation of Functionals by Neural Network without Curse of Dimensionalitycited by 8
  15. 2024Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networkscited by 8
  16. 2021A spectral-based analysis of the separation between two-layer neural networks and linear methodscited by 7
  17. 2023Infinite-dimensional reservoir computingcited by 7
  18. 2024How Transformers Get Rich: Approximation and Dynamics Analysiscited by 7
  19. 2021A Class of Dimension-free Metrics for the Convergence of Empirical Measurescited by 5
  20. 2022Universality and approximation bounds for echo state networks with random weightscited by 5
  21. 2023Generalization Ability of Wide Residual Networkscited by 5
  22. 2023Neural Hilbert Ladders: Multi-Layer Neural Networks in Function Spacecited by 5
  23. 2025Deep Neural Networks with General Activations: Super-Convergence in Sobolev Normscited by 5
  24. 2022Optimal bump functions for shallow ReLU networks: Weight decay, depth separation and the curse of dimensionalitycited by 4
  25. 2023Reinforcement Learning with Function Approximation: From Linear to Nonlinearcited by 4
  26. 2022A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networkscited by 3
  27. 2022Gradient-enhanced deep neural network approximationscited by 3
  28. 2022Why the pseudo label based semi-supervised learning algorithm is effective?cited by 3
  29. 2023On Excess Risk Convergence Rates of Neural Network Classifierscited by 3
  30. 2023Optimal Deep Neural Network Approximation for Korobov Functions with respect to Sobolev Normscited by 3
  31. 2023Minimum norm interpolation by perceptra: Explicit regularization and implicit biascited by 3
  32. 2021Nonasymptotic theory for two-layer neural networks: Beyond the bias-variance trade-offcited by 2
  33. 2022Qualitative neural network approximation over R and C: Elementary proofs for analytic and polynomial activationcited by 2
  34. 2023Delay-SDE-net: A deep learning approach for time series modelling with memory and uncertainty estimatescited by 2
  35. 2023Embedding Inequalities for Barron-type Spacescited by 2
  36. 2024Approximation Bounds for Recurrent Neural Networks with Application to Regressioncited by 2
  37. 2024Generative Feature Training of Thin 2-Layer Networkscited by 2
  38. 2024Mirror Descent on Reproducing Kernel Banach Spacescited by 2

50 most cited of 76 All 76 (JSON)