# On the rate of convergence of fully connected very deep neural network regression estimates

arXiv: https://arxiv.org/abs/1908.11133 · HTML: https://mlsanity.com/paper/1908.11133/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Aug 2019.

## Built on by (most cited first, 14 of 14)

- [Convergence rates of deep ReLU networks for multiclass classification](https://mlsanity.com/paper/2108.00969/) (2021)
- [Precision Machine Learning](https://mlsanity.com/paper/2210.13447/) (2022)
- [A likelihood approach to nonparametric estimation of a singular distribution using deep generative models](https://mlsanity.com/paper/2105.04046/) (2021)
- [How Neural Networks Learn the Support is an Implicit Regularization Effect of SGD](https://mlsanity.com/paper/2406.11110/) (2024)
- [Calibrating multi-dimensional complex ODE from noisy data via deep neural networks](https://mlsanity.com/paper/2106.03591/) (2021)
- [Estimation of a regression function on a manifold by fully connected deep neural networks](https://mlsanity.com/paper/2107.09532/) (2021)
- [Analysis of convolutional neural network image classifiers in a rotationally symmetric model](https://mlsanity.com/paper/2205.05500/) (2022)
- [Intrinsic and extrinsic deep learning on manifolds](https://mlsanity.com/paper/2302.08606/) (2023)
- [Analysis of the expected L2 error of an over-parametrized deep neural network estimate learned by gradient descent without regularization](https://mlsanity.com/paper/2311.14609/) (2023)
- [Confidence Interval Construction and Conditional Variance Estimation with Dense ReLU Networks](https://mlsanity.com/paper/2412.20355/) (2024)
- [Analysis of convolutional neural network image classifiers in a hierarchical max-pooling model with additional local pooling](https://mlsanity.com/paper/2106.05233/) (2021)
- [Semiparametric Regression for Spatial Data via Deep Learning](https://mlsanity.com/paper/2301.03747/) (2023)
- [On the rate of convergence of an over-parametrized Transformer classifier learned by gradient descent](https://mlsanity.com/paper/2312.17007/) (2023)
- [Soft Label PU Learning](https://mlsanity.com/paper/2405.01990/) (2024)

## Abstract (arXiv)

Recent results in nonparametric regression show that deep learning, i.e., neural network estimates with many hidden layers, are able to circumvent the so-called curse of dimensionality in case that suitable restrictions on the structure of the regression function hold. One key feature of the neural networks used in these results is that their network architecture has a further constraint, namely the network sparsity. In this paper we show that we can get similar results also for least squares estimates based on simple fully connected neural networks with ReLU activation functions. Here either the number of neurons per hidden layer is fixed and the number of hidden layers tends to infinity suitably fast for sample size tending to infinity, or the number of hidden layers is bounded by some logarithmic factor in the sample size and the number of neurons per hidden layer tends to infinity suitably fast for sample size tending to infinity. The proof is based on new approximation results concerning deep neural networks.