On the rate of convergence of fully connected very deep neural network regression estimates
Not on the map. Why?
Abstract (arXiv)
Recent results in nonparametric regression show that deep learning, i.e., neural network estimates with many hidden layers, are able to circumvent the so-called curse of dimensionality in case that suitable restrictions on the structure of the regression function hold. One key feature of the neural networks used in these results is that their network architecture has a further constraint, namely the network sparsity. In this paper we show that we can get similar results also for least squares estimates based on simple fully connected neural networks with ReLU activation functions. Here either the number of neurons per hidden layer is fixed and the number of hidden layers tends to infinity suitably fast for sample size tending to infinity, or the number of hidden layers is bounded by some logarithmic factor in the sample size and the number of neurons per hidden layer tends to infinity suitably fast for sample size tending to infinity. The proof is based on new approximation results concerning deep neural networks.
Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Aug 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 5; 2022: 2; 2023: 4; 2024: 3.
14 papers.
Sort:
- 2021Convergence rates of deep ReLU networks for multiclass classificationcited by 21
- 2022Precision Machine Learningcited by 14
- 2021A likelihood approach to nonparametric estimation of a singular distribution using deep generative modelscited by 12
- 2024How Neural Networks Learn the Support is an Implicit Regularization Effect of SGDcited by 8
- 2021Calibrating multi-dimensional complex ODE from noisy data via deep neural networkscited by 6
- 2021Estimation of a regression function on a manifold by fully connected deep neural networkscited by 5
- 2022Analysis of convolutional neural network image classifiers in a rotationally symmetric modelcited by 4
- 2023Intrinsic and extrinsic deep learning on manifoldscited by 4
- 2023Analysis of the expected L2 error of an over-parametrized deep neural network estimate learned by gradient descent without regularizationcited by 4
- 2024Confidence Interval Construction and Conditional Variance Estimation with Dense ReLU Networkscited by 3
- 2021Analysis of convolutional neural network image classifiers in a hierarchical max-pooling model with additional local poolingcited by 2
- 2023Semiparametric Regression for Spatial Data via Deep Learningcited by 2
2 more
- 2024Confidence Interval Construction and Conditional Variance Estimation with Dense ReLU Networks29 Dec 2024
- 2024How Neural Networks Learn the Support is an Implicit Regularization Effect of SGD17 Jun 2024
- 2024Soft Label PU Learning3 May 2024
- 2023On the rate of convergence of an over-parametrized Transformer classifier learned by gradient descent28 Dec 2023
- 2023Analysis of the expected L2 error of an over-parametrized deep neural network estimate learned by gradient descent without regularization24 Nov 2023
- 2023Intrinsic and extrinsic deep learning on manifolds16 Feb 2023
- 2023Semiparametric Regression for Spatial Data via Deep Learning10 Jan 2023
- 2022Precision Machine Learning24 Oct 2022
- 2022Analysis of convolutional neural network image classifiers in a rotationally symmetric model11 May 2022
- 2021Convergence rates of deep ReLU networks for multiclass classification2 Aug 2021
- 2021Estimation of a regression function on a manifold by fully connected deep neural networks20 Jul 2021
- 2021Calibrating multi-dimensional complex ODE from noisy data via deep neural networks7 Jun 2021
- 2021Convergence rates of deep ReLU networks for multiclass classificationcited by 21
- 2022Precision Machine Learningcited by 14
- 2021A likelihood approach to nonparametric estimation of a singular distribution using deep generative modelscited by 12
- 2024How Neural Networks Learn the Support is an Implicit Regularization Effect of SGDcited by 8
- 2021Calibrating multi-dimensional complex ODE from noisy data via deep neural networkscited by 6
- 2021Estimation of a regression function on a manifold by fully connected deep neural networkscited by 5
- 2022Analysis of convolutional neural network image classifiers in a rotationally symmetric modelcited by 4
- 2023Intrinsic and extrinsic deep learning on manifoldscited by 4
- 2023Analysis of the expected L2 error of an over-parametrized deep neural network estimate learned by gradient descent without regularizationcited by 4
- 2024Confidence Interval Construction and Conditional Variance Estimation with Dense ReLU Networkscited by 3
- 2021Analysis of convolutional neural network image classifiers in a hierarchical max-pooling model with additional local poolingcited by 2
- 2023Semiparametric Regression for Spatial Data via Deep Learningcited by 2
12 most cited of 14 All 14 (JSON)
- 2024Confidence Interval Construction and Conditional Variance Estimation with Dense ReLU Networks29 Dec 2024
- 2024How Neural Networks Learn the Support is an Implicit Regularization Effect of SGD17 Jun 2024
- 2024Soft Label PU Learning3 May 2024
- 2023On the rate of convergence of an over-parametrized Transformer classifier learned by gradient descent28 Dec 2023
- 2023Analysis of the expected L2 error of an over-parametrized deep neural network estimate learned by gradient descent without regularization24 Nov 2023
- 2023Intrinsic and extrinsic deep learning on manifolds16 Feb 2023
- 2023Semiparametric Regression for Spatial Data via Deep Learning10 Jan 2023
- 2022Precision Machine Learning24 Oct 2022
- 2022Analysis of convolutional neural network image classifiers in a rotationally symmetric model11 May 2022
- 2021Convergence rates of deep ReLU networks for multiclass classification2 Aug 2021
- 2021Estimation of a regression function on a manifold by fully connected deep neural networks20 Jul 2021
- 2021Calibrating multi-dimensional complex ODE from noisy data via deep neural networks7 Jun 2021
12 newest of 14 All 14 (JSON)