mlsanityData to 14 Oct 2026

Learning Two-layer Neural Networks with Symmetric Inputs

arXiv 1810.06793 ↗First posted 16 Oct 2018cs.LG stat.MLcited by 22

Not on the map. Why?

Abstract (arXiv)

We give a new algorithm for learning a two-layer neural network under a general class of input distributions. Assuming there is a ground-truth two-layer network y = A σ(Wx) + xi, where A,W are weight matrices, xi represents noise, and the number of neurons in the hidden layer is no larger than the input or output, our algorithm is guaranteed to recover the parameters A,W of the ground-truth network. The only requirement on the input x is that it is symmetric, which still allows highly complicated and structured input. Our algorithm is based on the method-of-moments framework and extends several results in tensor decompositions. We use spectral algorithms to avoid the complicated non-convex optimization in learning neural networks. Experiments show that our algorithm can robustly learn the ground-truth neural network with a small number of samples for many symmetric input distributions.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 4; 2022: 3; 2023: 7; 2024: 2; 2025: 6.

22 papers.

Sort:

By area:

  1. 2022C-Mixup: Improving Generalization in Regressioncited by 57
  2. 2021A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Networkcited by 47
  3. 2024Provably learning a multi-head attention layercited by 21
  4. 2021Efficient Algorithms for Learning Depth-2 Neural Networks with General ReLU Activationscited by 15
  5. 2023Learning Narrow One-Hidden-Layer ReLU Networkscited by 14
  6. 2025Identifiability of Deep Polynomial Neural Networkscited by 12
  7. 2023Computational Complexity of Learning Neural Networks: Smoothness and Degeneracycited by 10
  8. 2025Robust Learning of Multi-index Models via Iterative Subspace Approximationcited by 9
  9. 2021Scaffolding Setscited by 8
  10. 2023A faster and simpler algorithm for learning shallow networkscited by 7
  11. 2021Efficiently Learning Any One Hidden Layer ReLU Network From Queriescited by 6
  12. 2024Implicit High-Order Moment Tensor Estimation and Learning Latent Variable Modelscited by 6
10 more
  1. 2022Global Convergence of SGD On Two Layer Neural Netscited by 5
  2. 2023Efficiently Learning One-Hidden-Layer ReLU Networks via Schur Polynomialscited by 4
  3. 2025Learning Neural Networks with Distribution Shift: Efficiently Certifiable Guaranteescited by 4
  4. 2022Overparameterized ReLU Neural Networks Learn the Simplest Models: Neural Isometry and Exact Recoverycited by 3
  5. 2023Global Convergence of SGD For Logistic Loss on Two Layer Neural Netscited by 3
  6. 2023Most Neural Networks Are Almost Learnablecited by 2
  7. 2023Polynomial-Time Solutions for ReLU Network Training: A Complexity Classification via Max-Cut and Zonotopescited by 2
  8. 2025Low-Rank Tensor Decompositions for the Theory of Neural Networkscited by 2
  9. 2025Langevin Monte-Carlo Provably Learns Depth Two Neural Nets at Any Size and Datacited by 1
  10. 2025Provably Extracting the Features from a General Superpositioncited by 1