mlsanityData to 7 Oct 2026

Can Shallow Neural Networks Beat the Curse of Dimensionality? A mean field training perspective

arXiv 2005.10815 ↗First posted 21 May 2020cs.LG math.AP stat.MLcited by 22

Not on the map. Why?

Abstract (arXiv)

We prove that the gradient descent training of a two-layer neural network on empirical or population risk may not decrease population risk at an order faster than t-4/(d-2) under mean field scaling. Thus gradient descent training for fitting reasonably smooth, but truly high-dimensional data may be subject to the curse of dimensionality. We present numerical evidence that gradient descent training with general Lipschitz target functions becomes slower and slower as the dimension increases, but converges at approximately the same rate in all dimensions when the target function lies in the natural function space for two-layer ReLU networks.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 5; 2023: 4; 2024: 4; 2025: 5; 2026: 2.

22 papers.

Sort:

  1. 2021Learning the solution operator of parametric partial differential equations with physics-informed DeepOnetscited by 537
  2. 2023Can Physics-Informed Neural Networks beat the Finite Element Method?cited by 68
  3. 2022Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networkscited by 26
  4. 2023Sampling weights of deep neural networkscited by 20
  5. 2022On Feature Learning in Neural Networks with Global Convergence Guaranteescited by 12
  6. 2024Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networkscited by 8
  7. 2024Spectral Informed Neural Network: An Efficient and Low-Memory PINNcited by 6
  8. 2022Intrinsic Dimension for Large-Scale Geometric Learningcited by 4
  9. 2023On adversarial robustness and the use of Wasserstein ascent-descent dynamics to enforce itcited by 4
  10. 2022A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networkscited by 3
  11. 2022Proximal Mean Field Learning in Shallow Neural Networkscited by 1
  12. 2024Quantum Multi-Agent Reinforcement Learning for Cooperative Mobile Access in Space-Air-Ground Integrated Networkscited by 1
10 more
  1. 2024Sinc Kolmogorov-Arnold network and its application for solving PDEs with singularitiescited by 1
  2. 2025Curse of Dimensionality in Neural Network Optimizationcited by 1
  3. 2026Elliptic Regularity Theory in Barron Spaces and Applications to the Deep Ritz Methodcited by 1
  4. 2021A Neurorobotics Approach to Behaviour Selection based on Human Activity Recognitioncited by 0
  5. 2023Evolution of MAC Protocols in the Machine Learning Decade: A Comprehensive Surveycited by 0
  6. 2025PINNs Algorithmic Framework for Simulation of Nonlinear Burgers' Type Modelscited by 0
  7. 2025Quasi-Random Physics-informed Neural Networkscited by 0
  8. 2025AI paradigm for solving differential equations: first-principles data generation and scale-dilation operator AI solvercited by 0
  9. 2025Multi-Scale Finite Expression Method for PDEs with Oscillatory Solutions on Complex Domainscited by 0
  10. 2026Deep Neural networks for solving high-dimensional parabolic partial differential equationscited by 0