mlsanityData to 7 Oct 2026

Provable approximation properties for deep neural networks

arXiv 1509.07385 ↗First posted 24 Sep 2015stat.ML cs.LG cs.NEcited by 60

Not on the map. Why?

Abstract (arXiv)

We discuss approximation of functions using deep neural nets. Given a function f on a d-dimensional manifold Gamma subset ℝm, we construct a sparsely-connected depth-4 neural network and bound its error in approximating f. The size of the network depends on dimension and curvature of the manifold Gamma, the complexity of f, in terms of its wavelet description, and only weakly on the ambient dimension m. Essentially, our network computes wavelet functions, which are computed from Rectified Linear Units (ReLU)

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Sep 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 21; 2023: 13; 2024: 6; 2025: 9; 2026: 3.

60 papers.

Sort:

By area:

  1. 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 226
  2. 2022On the Optimization Landscape of Neural Collapse under MSE Loss: Global Optimality with Unconstrained Featurescited by 97
  3. 2022Are All Losses Created Equal: A Neural Collapse Perspectivecited by 78
  4. 2022Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifoldcited by 50
  5. 2022Deep Nonparametric Estimation of Operators between Infinite Dimensional Spacescited by 45
  6. 2022Low Dimensional Invariant Embeddings for Universal Geometric Learningcited by 38
  7. 2022Deep Learning Methods for Partial Differential Equations and Related Parameter Identification Problemscited by 34
  8. 2021Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networkscited by 32
  9. 2023Sumformer: Universal Approximation for Efficient Transformerscited by 30
  10. 2022Why Robust Generalization in Deep Learning is Difficult: Perspective of Expressive Powercited by 23
  11. 2021Theory of Deep Convolutional Neural Networks III: Approximating Radial Functionscited by 22
  12. 2022Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformercited by 22
38 more
  1. 2025Understanding In-Context Learning on Structured Manifolds: Bridging Attention to Kernel Methodscited by 13
  2. 2022Sample Complexity of Nonparametric Off-Policy Evaluation on Low-Dimensional Manifolds using Deep Networkscited by 12
  3. 2022Effects of Data Geometry in Early Deep Learningcited by 12
  4. 2023Rates of Approximation by ReLU Shallow Neural Networkscited by 12
  5. 2024Cross Entropy versus Label Smoothing: A Neural Collapse Perspectivecited by 11
  6. 2023Deep Nonparametric Estimation of Intrinsic Data Structures by Chart Autoencoders: Generalization Error and Robustnesscited by 10
  7. 2022Deep neural network expressivity for optimal stopping problemscited by 9
  8. 2021Stable Recovery of Entangled Weights: Towards Robust Identification of Deep Neural Networks from Minimal Samplescited by 8
  9. 2022Semi-Supervised Manifold Learning with Complexity Decoupled Chart Autoencoderscited by 8
  10. 2023Approximation of Nonlinear Functionals Using Deep ReLU Networkscited by 8
  11. 2023Implicit Neural Representations and the Algebra of Complex Waveletscited by 8
  12. 2021Deep Neural Networks with ReLU-Sine-Exponential Activations Break Curse of Dimensionality in Approximation on H"older Classcited by 7
  13. 2022Approximation results for Gradient Descent trained Shallow Neural Networks in 1dcited by 7
  14. 2024Learning smooth functions in high dimensions: from sparse polynomials to deep neural networkscited by 7
  15. 2022Side Effects of Learning from Low-dimensional Data Embedded in a Euclidean Spacecited by 6
  16. 2022Benefits of Overparameterized Convolutional Residual Networks: Function Approximation under Smoothness Constraintcited by 6
  17. 2021Generalization Performance of Empirical Risk Minimization on Over-parameterized Deep ReLU Netscited by 4
  18. 2021Approximation of functions with one-bit neural networkscited by 4
  19. 2022Theoretical Exploration of Solutions of Feedforward ReLU Networkscited by 4
  20. 2023Deep Convolutional Neural Networks with Zero-Padding: Feature Extraction and Learningcited by 4
  21. 2023Classification of Data Generated by Gaussian Mixture Models Using Deep ReLU Networkscited by 4
  22. 2025On the Expressive Power of Mixture-of-Experts for Structured Complex Taskscited by 4
  23. 2022Deep neural networks can stably solve high-dimensional, noisy, non-linear inverse problemscited by 3
  24. 2023Linear Regression on Manifold Structured Data: the Impact of Extrinsic Geometry on Solutionscited by 3
  25. 2024Just How Flexible are Neural Networks in Practice?cited by 3
  26. 2025Bi-Lipschitz Ansatz for Anti-Symmetric Functionscited by 3
  27. 2025Transformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization Insightscited by 3
  28. 2025Geometry of Neural Reinforcement Learning in Continuous State and Action Spacescited by 3
  29. 2021A Dimensionality Reduction Approach for Convolutional Neural Networkscited by 2
  30. 2022Limitations of neural network training due to numerical instability of backpropagationcited by 2
  31. 2023Approximation Results for Gradient Descent trained Neural Networkscited by 2
  32. 2024Convergence Analysis for Deep Sparse Coding via Convolutional Neural Networkscited by 2
  33. 2022Improved Overparametrization Bounds for Global Convergence of Stochastic Gradient Descent for Shallow Neural Networkscited by 1
  34. 2022Universal Solutions of Feedforward ReLU Networks for Interpolationscited by 1
  35. 2023Statistical learning by sparse deep neural networkscited by 1
  36. 2024Component-based Sketching for Deep ReLU Netscited by 1
  37. 2026Quantitative Approximation Rates for Group Equivariant Learningcited by 1
  38. 2022On the Geometry of Reinforcement Learning in Continuous State and Action Spacescited by 0

50 most cited of 60 All 60 (JSON)