mlsanityData to 14 Oct 2026

Mean-Field Analysis for Learning Subspace-Sparse Polynomials with Gaussian Input

arXiv 2402.08948 ↗First posted 14 Feb 2024cs.LG math.APcited by 1

Not on the map. Why?

Abstract (arXiv)

In this work, we study the mean-field flow for learning subspace-sparse polynomials using stochastic gradient descent and two-layer neural networks, where the input distribution is standard Gaussian and the output only depends on the projection of the input onto a low-dimensional subspace. We establish a necessary condition for SGD-learnability, involving both the characteristics of the target function and the expressiveness of the activation function. In addition, we prove that the condition is almost sufficient, in the sense that a condition slightly stronger than the necessary condition can guarantee the exponential decay of the loss functional to zero.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 21 of its 31 references that are in the map. First posted on arXiv 14 Feb 2024.

Builds on

21 of 31 references.

References · 21

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  2. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
  3. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 659
  4. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 466
  5. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 426
  6. 2016Deep Learning without Poor Local Minimacited by 234
  7. 2019Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limitcited by 187
  8. 2023SGD learning on neural networks: leap complexity and saddle-to-saddle dynamicscited by 143
  9. 2022The merged-staircase property: a necessary and nearly sufficient condition for SGD learning of sparse functions on two-layer neural networkscited by 142
  10. 2018Mean Field Analysis of Neural Networks: A Central Limit Theoremcited by 107
  11. 2018Trainability and Accuracy of Neural Networks: An Interacting Particle System Approachcited by 106
  12. 2017Spurious Local Minima are Common in Two-Layer ReLU Neural Networkscited by 104
9 more
  1. 2019On the Power and Limitations of Random Features for Understanding Neural Networkscited by 103
  2. 2023How Two-Layer Neural Networks Learn, One (Giant) Step at a Timecited by 68
  3. 2021The staircase property: How hierarchical structure can guide deep learningcited by 62
  4. 2023On Learning Gaussian Multi-index Models with Gradient Flowcited by 57
  5. 2024The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponentscited by 52
  6. 2017Depth Creates No Bad Local Minimacited by 29
  7. 2019Mean Field Limit of the Learning Dynamics of Multilayer Neural Networkscited by 25
  8. 2023Beyond NTK with Vanilla Gradient Descent: A Mean-Field Analysis of Neural Networks with Polynomial Width, Samples, and Timecited by 20
  9. 2018Depth with Nonlinearity Creates No Bad Local Minima in ResNetscited by 17

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Robust Feature Learning for Multi-Index Models in High Dimensionscited by 5