# Can Shallow Neural Networks Beat the Curse of Dimensionality? A mean field training perspective

arXiv: https://arxiv.org/abs/2005.10815 · HTML: https://mlsanity.com/paper/2005.10815/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2020.

## Built on by (most cited first, 22 of 22)

- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Can Physics-Informed Neural Networks beat the Finite Element Method?](https://mlsanity.com/paper/2302.04107/) (2023)
- [Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networks](https://mlsanity.com/paper/2205.02902/) (2022)
- [Sampling weights of deep neural networks](https://mlsanity.com/paper/2306.16830/) (2023)
- [On Feature Learning in Neural Networks with Global Convergence Guarantees](https://mlsanity.com/paper/2204.10782/) (2022)
- [Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks](https://mlsanity.com/paper/2404.18769/) (2024)
- [Spectral Informed Neural Network: An Efficient and Low-Memory PINN](https://mlsanity.com/paper/2408.16414/) (2024)
- [Intrinsic Dimension for Large-Scale Geometric Learning](https://mlsanity.com/paper/2210.05301/) (2022)
- [On adversarial robustness and the use of Wasserstein ascent-descent dynamics to enforce it](https://mlsanity.com/paper/2301.03662/) (2023)
- [A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networks](https://mlsanity.com/paper/2210.16286/) (2022)
- [Quantum Multi-Agent Reinforcement Learning for Cooperative Mobile Access in Space-Air-Ground Integrated Networks](https://mlsanity.com/paper/2406.16994/) (2024)
- [Proximal Mean Field Learning in Shallow Neural Networks](https://mlsanity.com/paper/2210.13879/) (2022)
- [Sinc Kolmogorov-Arnold network and its application for solving PDEs with singularities](https://mlsanity.com/paper/2410.04096/) (2024)
- [Curse of Dimensionality in Neural Network Optimization](https://mlsanity.com/paper/2502.05360/) (2025)
- [Deep Neural networks for solving high-dimensional parabolic partial differential equations](https://mlsanity.com/paper/2601.13256/) (2026)
- [Elliptic Regularity Theory in Barron Spaces and Applications to the Deep Ritz Method](https://mlsanity.com/paper/2607.25100/) (2026)
- [A Neurorobotics Approach to Behaviour Selection based on Human Activity Recognition](https://mlsanity.com/paper/2107.12540/) (2021)
- [Evolution of MAC Protocols in the Machine Learning Decade: A Comprehensive Survey](https://mlsanity.com/paper/2302.13876/) (2023)
- [PINNs Algorithmic Framework for Simulation of Nonlinear Burgers' Type Models](https://mlsanity.com/paper/2506.12922/) (2025)
- [Quasi-Random Physics-informed Neural Networks](https://mlsanity.com/paper/2507.08121/) (2025)
- [AI paradigm for solving differential equations: first-principles data generation and scale-dilation operator AI solver](https://mlsanity.com/paper/2507.23141/) (2025)
- [Multi-Scale Finite Expression Method for PDEs with Oscillatory Solutions on Complex Domains](https://mlsanity.com/paper/2510.22497/) (2025)

## Abstract (arXiv)

We prove that the gradient descent training of a two-layer neural network on empirical or population risk may not decrease population risk at an order faster than t-4/(d-2) under mean field scaling. Thus gradient descent training for fitting reasonably smooth, but truly high-dimensional data may be subject to the curse of dimensionality. We present numerical evidence that gradient descent training with general Lipschitz target functions becomes slower and slower as the dimension increases, but converges at approximately the same rate in all dimensions when the target function lies in the natural function space for two-layer ReLU networks.