mlsanityData to 7 Oct 2026

Statistically guided deep learning

arXiv 2504.08489 ↗First posted 11 Apr 2025math.ST cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

We present a theoretically well-founded deep learning algorithm for nonparametric regression. It uses over-parametrized deep neural networks with logistic activation function, which are fitted to the given data via gradient descent. We propose a special topology of these networks, a special random initialization of the weights, and a data-dependent choice of the learning rate and the number of gradient descent steps. We prove a theoretical bound on the expected L2 error of this estimate, and illustrate its finite sample size performance by applying it to simulated data. Our results show that a theoretical analysis of deep learning which takes into account simultaneously optimization, generalization and approximation can result in a new deep learning estimate which has an improved finite sample performance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 42 references that are in the map. First posted on arXiv 11 Apr 2025.

Builds on

26 of 42 references.

References · 26

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  3. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  4. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  5. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 572
  6. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 463
  7. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  8. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 422
  9. 2017Size-Independent Sample Complexity of Neural Networkscited by 289
  10. 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 165
  11. 2020Deep Network Approximation for Smooth Functionscited by 163
  12. 2023SGD learning on neural networks: leap complexity and saddle-to-saddle dynamicscited by 142
14 more
  1. 2018Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networkscited by 139
  2. 2020Online stochastic gradient descent on non-convex losses from high-dimensional inferencecited by 119
  3. 2019Finite Depth and Width Corrections to the Neural Tangent Kernelcited by 114
  4. 2022Learning Single-Index Models with Shallow Neural Networkscited by 92
  5. 2019The phase diagram of approximation rates for deep neural networkscited by 86
  6. 2020A Rigorous Framework for the Mean Field Limit of Multilayer Neural Networkscited by 68
  7. 2015Learning with Square Loss: Localization through Offset Rademacher Complexitycited by 37
  8. 2020Optimal Rates for Averaged Stochastic Gradient Descent under Neural Tangent Kernel Regimecited by 35
  9. 2019Gradient Descent Finds Global Minima for Generalizable Deep Neural Networks of Practical Sizescited by 29
  10. 2019Generalization Bounds for Convolutional Neural Networkscited by 21
  11. 2020Approximating smooth functions by deep neural networks with sigmoid activation functioncited by 18
  12. 2023Deep Neural Networks for Nonparametric Interaction Models with Diverging Dimensioncited by 15
  13. 2022Generalization Error Bounds for Deep Neural Networks Trained by SGDcited by 14
  14. 2023Analysis of the expected L2 error of an over-parametrized deep neural network estimate learned by gradient descent without regularizationcited by 4

Built on by

Not cited yet.