mlsanityData to 13 Oct 2026

Nonparametric Regression on Low-Dimensional Manifolds using Deep ReLU Networks : Function Approximation and Statistical Recovery

arXiv 1908.01842 ↗First posted 5 Aug 2019cs.LG stat.MLcited by 73

Not on the map. Why?

Abstract (arXiv)

Real world data often exhibit low-dimensional geometric structures, and can be viewed as samples near a low-dimensional manifold. This paper studies nonparametric regression of H"older functions on low-dimensional manifolds using deep ReLU networks. Suppose n training data are sampled from a H"older function in Hs,α supported on a d-dimensional Riemannian manifold isometrically embedded in ℝD, with sub-gaussian noise. A deep ReLU network architecture is designed to estimate the underlying function from the training data. The mean squared error of the empirical estimator is proved to converge in the order of n^-frac2(s+α)2(s+α) + dlog3 n. This result shows that deep ReLU networks give rise to a fast convergence rate depending on the data intrinsic dimension d, which is usually much smaller than the ambient dimension D. It therefore demonstrates the adaptivity of deep ReLU networks to low-dimensional geometric structures of data, and partially explains the power of deep ReLU networks in tackling high-dimensional data with low-dimensional geometric structures.

Key facts. Cited by 73 papers in the map (data through 13 Oct 2026). First posted on arXiv 5 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 5; 2022: 10; 2023: 16; 2024: 19; 2025: 9; 2026: 14.

73 papers.

Sort:

By area:

  1. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Datacited by 172
  2. 2022Neural Networks Efficiently Learn Low-Dimensional Representations with SGDcited by 63
  3. 2022Deep Nonparametric Estimation of Operators between Infinite Dimensional Spacescited by 45
  4. 2021Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networkscited by 32
  5. 2023Approximation and Estimation Ability of Transformers for Sequence-to-Sequence Functions with Infinite Dimensional Inputcited by 32
  6. 2024Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Datacited by 30
  7. 2023Nonparametric regression using over-parameterized shallow ReLU neural networkscited by 24
  8. 2024Diffusion Model for Data-Driven Black-Box Optimizationcited by 24
  9. 2024A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Modelscited by 20
  10. 2023Deep Operator Learning Lessens the Curse of Dimensionality for PDEscited by 18
  11. 2024Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Studycited by 14
  12. 2025Understanding In-Context Learning on Structured Manifolds: Bridging Attention to Kernel Methodscited by 14
38 more
  1. 2023On Deep Generative Models for Approximation and Estimation of Distributions on Manifoldscited by 13
  2. 2023Provable Guarantees for Neural Networks via Gradient Feature Learningcited by 13
  3. 2024On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networkscited by 13
  4. 2021A likelihood approach to nonparametric estimation of a singular distribution using deep generative modelscited by 12
  5. 2022Sample Complexity of Nonparametric Off-Policy Evaluation on Low-Dimensional Manifolds using Deep Networkscited by 12
  6. 2024Generalization Error Guaranteed Auto-Encoder-Based Nonlinear Model Reduction for Operator Learningcited by 12
  7. 2023Deep Nonparametric Estimation of Intrinsic Data Structures by Chart Autoencoders: Generalization Error and Robustnesscited by 10
  8. 2021Stationary Density Estimation of Ito Diffusions Using Deep Learningcited by 9
  9. 2023Differentiable Neural Networks with RePU Activation: with Applications to Score Estimation and Isotonic Regressioncited by 9
  10. 2023Double-well Net for Image Segmentationcited by 7
  11. 2024Learning smooth functions in high dimensions: from sparse polynomials to deep neural networkscited by 7
  12. 2021Deep Networks Provably Classify Data on Curvescited by 6
  13. 2022Benefits of Overparameterized Convolutional Residual Networks: Function Approximation under Smoothness Constraintcited by 6
  14. 2024Regularized DeepIV with Model Selectioncited by 6
  15. 2024How DNNs break the Curse of Dimensionality: Compositionality and Symmetry Learningcited by 6
  16. 2022A Manifold Two-Sample Test Study: Integral Probability Metric with Neural Networkscited by 5
  17. 2025Provable Sample-Efficient Transfer Learning Conditional Diffusion Models via Representation Learningcited by 5
  18. 2021Non-asymptotic Excess Risk Bounds for Classification with Deep Convolutional Neural Networkscited by 4
  19. 2022Fitting an immersed submanifold to data via Sussmann's orbit theoremcited by 4
  20. 2022Predicting Out-of-Domain Generalization with Neighborhood Invariancecited by 4
  21. 2023Intrinsic and extrinsic deep learning on manifoldscited by 4
  22. 2024Deep Transfer Learning: Model Framework and Error Analysiscited by 4
  23. 2025Approximation Bounds for Transformer Networks with Application to Regressioncited by 4
  24. 2022Deep neural networks can stably solve high-dimensional, noisy, non-linear inverse problemscited by 3
  25. 2022Estimation of Non-Crossing Quantile Regression Process with Deep ReQU Neural Networkscited by 3
  26. 2023Solving PDEs on Spheres with Physics-Informed Convolutional Neural Networkscited by 3
  27. 2024Deep Neural Networks are Adaptive to Function Regularity and Data Distribution in Approximation and Estimationcited by 3
  28. 2024Neural Networks Generalize on Low Complexity Datacited by 3
  29. 2025Coefficient-to-Basis Network: A Fine-Tunable Operator Learning Framework for Inverse Problems with Adaptive Discretizations and Theoretical Guaranteescited by 3
  30. 2025Transformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization Insightscited by 3
  31. 2026Latent Autoencoder Ensemble Kalman Filter for Nonlinear Data assimilationcited by 3
  32. 2026Generalization Bounds and Statistical Guarantees for Multi-Task and Multiple Operator Learning with MNO Networkscited by 3
  33. 2026Approximation and learning of anisotropic and mixed smooth functions by deep ReLU neural networkscited by 3
  34. 2022Resource-Efficient Invariant Networks: Exponential Gains by Unrolled Optimizationcited by 2
  35. 2023Nonparametric Classification on Low Dimensional Manifolds using Overparameterized Convolutional Residual Networkscited by 2
  36. 2023Spectral Neural Networks: Approximation Theory and Optimization Landscapecited by 2
  37. 2024Defining Neural Network Architecture through Polytope Structures of Datasetcited by 2
  38. 2024Causal inference through multi-stage learning and doubly robust deep neural networkscited by 2

50 most cited of 73 All 73 (JSON)