mlsanityData to 7 Oct 2026

Estimation of a regression function on a manifold by fully connected deep neural networks

arXiv 2107.09532 ↗First posted 20 Jul 2021math.ST stat.ML stat.THcited by 5

Not on the map. Why?

Abstract (arXiv)

Estimation of a regression function from independent and identically distributed data is considered. The L2 error with integration with respect to the distribution of the predictor variable is used as the error criterion. The rate of convergence of least squares estimates based on fully connected spaces of deep neural networks with ReLU activation function is analyzed for smooth regression functions. It is shown that in case that the distribution of the predictor variable is concentrated on a manifold, these estimates achieve a rate of convergence which depends on the dimension of the manifold and not on the number of components of the predictor variable.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 28 of its 63 references that are in the map. First posted on arXiv 20 Jul 2021.

Builds on

28 of 63 references.

References · 28

  1. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  2. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 666
  3. 2018Rethinking the Value of Network Pruningcited by 595
  4. 2016Error bounds for approximations with deep ReLU networkscited by 567
  5. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 562
  6. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  7. 2014Identifying and attacking the saddle point problem in high-dimensional non-convex optimizationcited by 286
  8. 2014The Loss Surfaces of Multilayer Networkscited by 278
  9. 2018Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Datacited by 260
  10. 2016Deep Learning without Poor Local Minimacited by 233
  11. 2015The Power of Depth for Feedforward Neural Networkscited by 181
  12. 2020Deep Network Approximation for Smooth Functionscited by 163
16 more
  1. 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 160
  2. 2018On the Power of Over-parametrization in Neural Networks with Quadratic Activationcited by 102
  3. 2019The phase diagram of approximation rates for deep neural networkscited by 86
  4. 2019Deep ReLU network approximation of functions on a manifoldcited by 73
  5. 2019Theoretical Issues in Deep Networks: Approximation, Optimization and Generalizationcited by 72
  6. 2016Deep vs. shallow networks : An approximation theory perspectivecited by 63
  7. 2018Deep Neural Networks for Estimation and Inferencecited by 62
  8. 2019The Difficulty of Training Sparse Neural Networkscited by 62
  9. 2018Deep Neural Networks Learn Non-Smooth Functions Effectivelycited by 57
  10. 2019Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov spacecited by 46
  11. 2016Deep nets for local manifold learningcited by 26
  12. 2018A comparison of deep networks with ReLU activation function and linear spline-type methodscited by 18
  13. 2020Approximating smooth functions by deep neural networks with sigmoid activation functioncited by 18
  14. 2019On the rate of convergence of fully connected very deep neural network regression estimatescited by 14
  15. 2019Adaptive Approximation and Generalization of Deep Neural Network with Intrinsic Dimensionalitycited by 9
  16. 2013Universal Approximation Depth and Errors of Narrow Belief Networks with Discrete Unitscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 1; 2026: 1.

5 papers.

  1. 2024Scalable Subsampling Inference for Deep Neural Networkscited by 5
  2. 2024A Likelihood Based Approach to Distribution Regression Using Conditional Deep Generative Modelscited by 3
  3. 2025Adjustment for Confounding using Pre-Trained Representationscited by 3
  4. 2023Mercer Large-Scale Kernel Machines from Ridge Function Perspectivecited by 0
  5. 2026Rethinking Intrinsic Dimension Estimation in Neural Representationscited by 0