mlsanityData to 7 Oct 2026

Learning a Single Neuron with Gradient Methods

arXiv 2001.05205 ↗First posted 15 Jan 2020cs.LG cs.NE stat.MLcited by 50

Not on the map. Why?

Abstract (arXiv)

We consider the fundamental problem of learning a single neuron x mapstoσ(w^top x) using standard gradient methods. As opposed to previous works, which considered specific (and not always realistic) input distributions and activation functions σ(·), we ask whether a more general result is attainable, under milder assumptions. On the one hand, we show that some assumptions on the distribution and the activation function are necessary. On the other hand, we prove positive guarantees under mild assumptions, which go beyond those studied in the literature so far. We also point out and study the challenges in further strengthening and generalizing our results.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 10; 2023: 14; 2024: 6; 2025: 5; 2026: 6.

50 papers.

Sort:

  1. 2022Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limitcited by 184
  2. 2022The Implicit Bias of Benign Overfittingcited by 38
  3. 2021Early-stopped neural networks are consistentcited by 34
  4. 2022Random Feature Amplification: Feature Learning and Generalization in Neural Networkscited by 33
  5. 2023Gradient-Based Feature Learning under Structured Datacited by 32
  6. 2023Over-Parameterization Exponentially Slows Down Gradient Descent for Learning a Single Neuroncited by 25
  7. 2021Proxy Convexity: A Unified Framework for the Analysis of Neural Networks Trained by Gradient Descentcited by 21
  8. 2022Learning a Single Neuron with Adversarial Label Noise via Gradient Descentcited by 20
  9. 2021Learning a Single Neuron with Bias Using Gradient Descentcited by 17
  10. 2021Optimization-Based Separations for Neural Networkscited by 17
  11. 2023On Single Index Models beyond Gaussian Datacited by 17
  12. 2021On Learnability via Gradient Method for Two-Layer ReLU Neural Networks in Teacher-Student Settingcited by 16
38 more
  1. 2023Learning a Neuron by a Shallow ReLU Network: Dynamics and Implicit Bias for Correlated Inputscited by 16
  2. 2024Masks, Signs, And Learning Rate Rewindingcited by 15
  3. 2022Learning a Single Neuron for Non-monotonic Activation Functionscited by 13
  4. 2022Active Learning for Single Neuron Models with Lipschitz Non-Linearitiescited by 12
  5. 2023Robustly Learning a Single Neuron via Sharpnesscited by 12
  6. 2021ReLU Regression with Massart Noisecited by 10
  7. 2022Excess Risk of Two-Layer ReLU Neural Networks in Teacher-Student Settings and its Superiority to Kernel Methodscited by 10
  8. 2023Finite-Sample Analysis of Learning High-Dimensional Single ReLU Neuroncited by 8
  9. 2023Should Under-parameterized Student Networks Copy or Average Teacher Weights?cited by 8
  10. 2025Low-dimensional Functions are Efficiently Learnable under Randomly Biased Distributionscited by 8
  11. 2022Early Stage Convergence and Global Convergence of Training Mildly Parameterized Neural Networkscited by 5
  12. 2022Magnitude and Angle Dynamics in Training Single ReLU Neuronscited by 5
  13. 2022SQ Lower Bounds for Learning Single Neurons with Massart Noisecited by 5
  14. 2023Continuized Acceleration for Quasar Convex Functions in Non-Convex Optimizationcited by 5
  15. 2023Symmetric Single Index Learningcited by 5
  16. 2023Max-affine regression via first-order methodscited by 3
  17. 2023The Effect of SGD Batch Size on Autoencoder Learning: Sparsity, Sharpness, and Feature Learningcited by 2
  18. 2024Learning a Single Neuron Robustly to Distributional Shifts and Adversarial Label Noisecited by 2
  19. 2026Deep Networks Learn Deep Hierarchical Modelscited by 2
  20. 2023Distribution-Independent Regression for Generalized Linear Models with Oblivious Corruptionscited by 1
  21. 2023The Local Landscape of Phase Retrieval Under Limited Samplescited by 1
  22. 2024RedEx: Beyond Fixed Representation Methods via Convex Optimizationcited by 1
  23. 2025Joint Learning in the Gaussian Single Index Modelcited by 1
  24. 2026Interactive Learning of Single-Index Models via Stochastic Gradient Descentcited by 1
  25. 2026When Both Layers Learn: Training Dynamics of Representing Linear Models via ReLU Networkscited by 1
  26. 2021Painless step size adaptation for SGDcited by 0
  27. 2021Neurons learn slower than they thinkcited by 0
  28. 2021Directional Convergence Analysis under Spherically Symmetric Distributioncited by 0
  29. 2023Asymptotically efficient adaptive identification under saturated output observationcited by 0
  30. 2024Bayesian Inference for Consistent Predictions in Overparameterized Nonlinear Regressioncited by 0
  31. 2024Disentangle Sample Size and Initialization Effect on Perfect Generalization for Single-Neuron Targetcited by 0
  32. 2024Online Non-Stationary Stochastic Quasar-Convex Optimizationcited by 0
  33. 2025A Derandomization Framework for Structure Discovery: Applications in Neural Networks and Beyondcited by 0
  34. 2025Block Coordinate Descent for Neural Networks Provably Finds Global Minimacited by 0
  35. 2025Universality of high-dimensional scaling limits of stochastic gradient descentcited by 0
  36. 2026Robust Learning of a Group DRO Neuroncited by 0
  37. 2026Singular perturbations and hierarchical learning in two-layer neural networkscited by 0
  38. 2026Two-Timescale Fine-tuning Provably Learns New Features for Two-Layer ReLU Networkscited by 0