mlsanityData to 7 Oct 2026

Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Review

arXiv 1611.00740 ↗First posted 2 Nov 2016cs.LGcited by 155

Not on the map. Why?

Abstract (arXiv)

The paper characterizes classes of functions for which deep learning can be exponentially better than shallow learning. Deep convolutional networks are a special case of these conditions, though weight sharing is not the main reason for their exponential advantage.

Key facts. Cited by 155 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 41; 2022: 22; 2023: 25; 2024: 22; 2025: 25; 2026: 20.

155 papers.

Sort:

By area:

  1. 2021Learning the solution operator of parametric partial differential equations with physics-informed DeepOnetscited by 537
  2. 2024A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Datacited by 88
  3. 2021Multi-Objective Loss Balancing for Physics-Informed Deep Learningcited by 86
  4. 2021On the approximation of functions by tanh neural networkscited by 81
  5. 2023How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Modelcited by 70
  6. 2021When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization?cited by 57
  7. 2021A novel meta-learning initialization method for physics-informed neural networkscited by 35
  8. 2023Deep networks for system identification: a Surveycited by 35
  9. 2022ExSpliNet: An interpretable and expressive spline-based neural networkcited by 33
  10. 2021Approximation and Learning with Deep Convolutional Models: a Kernel Perspectivecited by 31
  11. 2021Locality defeats the curse of dimensionality in convolutional teacher-student scenarioscited by 26
  12. 2024Probing the Latent Hierarchical Structure of Data via Diffusion Modelscited by 25
38 more
  1. 2022On the Number of Regions of Piecewise Linear Neural Networkscited by 21
  2. 2023Theoretical Analysis of Inductive Biases in Deep Convolutional Networkscited by 21
  3. 2021Reproducing Activation Function for Deep Learningcited by 19
  4. 2021Depth separation beyond radial functionscited by 18
  5. 2021Optimization-Based Separations for Neural Networkscited by 17
  6. 2023Solving Forward and Inverse Problems of Contact Mechanics using Physics-Informed Neural Networkscited by 17
  7. 2024U-Nets as Belief Propagation: Efficient Classification, Denoising, and Diffusion in Generative Hierarchical Modelscited by 17
  8. 2022What Can Be Learnt With Wide Convolutional Neural Networks?cited by 16
  9. 2023Classification with Deep Neural Networks and Logistic Losscited by 16
  10. 2021Approximation Properties of Deep ReLU CNNscited by 15
  11. 2022On the Spectral Bias of Convolutional Neural Tangent and Gaussian Process Kernelscited by 15
  12. 2024A Survey on Universal Approximation Theoremscited by 15
  13. 2021Layer Folding: Neural Network Depth Reduction using Activation Linearizationcited by 14
  14. 2021Convergence of Deep ReLU Networkscited by 14
  15. 2022Precision Machine Learningcited by 14
  16. 2021Neural Trees for Learning on Graphscited by 13
  17. 2021On the Sample Complexity of Learning under Invariance and Geometric Stabilitycited by 13
  18. 2021Convergence of Deep Convolutional Neural Networkscited by 13
  19. 2025Position: A Theory of Deep Learning Must Include Compositional Sparsitycited by 13
  20. 2021ReLU Deep Neural Networks from the Hierarchical Basis Perspectivecited by 12
  21. 2021Reverse Engineering the Neural Tangent Kernelcited by 12
  22. 2022A Kernel Perspective of Skip Connections in Convolutional Networkscited by 12
  23. 2021Stationary Density Estimation of Ito Diffusions Using Deep Learningcited by 9
  24. 2023Identifying Interpretable Visual Features in Artificial and Biological Neural Systemscited by 9
  25. 2025Neural Tangent Kernel Analysis to Probe Convergence in Physics-informed Neural Solvers: PIKANs vs. PINNscited by 9
  26. 2023Depth Separation with Multilayer Mean-Field Networkscited by 8
  27. 2023Expressivity and Approximation Properties of Deep Neural Networks with ReLUk Activationcited by 8
  28. 2021Deep Neural Networks with ReLU-Sine-Exponential Activations Break Curse of Dimensionality in Approximation on H"older Classcited by 7
  29. 2021Data-Driven Computational Methods for the Domain of Attraction and Zubov's Equationcited by 7
  30. 2022Approximation results for Gradient Descent trained Shallow Neural Networks in 1dcited by 7
  31. 2024Learning smooth functions in high dimensions: from sparse polynomials to deep neural networkscited by 7
  32. 2024How Deep Networks Learn Sparse and Hierarchical Data: the Sparse Random Hierarchy Modelcited by 7
  33. 2021A shallow neural model for relation predictioncited by 6
  34. 2021Distributional Offline Continuous-Time Reinforcement Learning with Neural Physics-Informed PDEs (SciPhy RL for DOCTR-L)cited by 6
  35. 2022Minimax Optimality (Probably) Doesn't Imply Distribution Learning for GANscited by 6
  36. 2022Gradient-Enhanced Physics-Informed Neural Networks for Power Systems Operational Supportcited by 6
  37. 2024Energy Dissipation Preserving Physics Informed Neural Network for Allen-Cahn Equationscited by 6
  38. 2025F-INR: Functional Tensor Decomposition for Implicit Neural Representationscited by 6

50 most cited of 155 All 155 (JSON)