mlsanityData to 14 Oct 2026

Quadratic Suffices for Over-parametrization via Matrix Chernoff Bound

arXiv 1906.03593 ↗First posted 9 Jun 2019cs.LG cs.AI stat.MLcited by 59

Not on the map. Why?

Abstract (arXiv)

We improve the over-parametrization size over two beautiful results [Li and Liang' 2018] and [Du, Zhai, Poczos and Singh' 2019] in deep learning theory.

Key facts. Cited by 59 papers in the map (data through 14 Oct 2026). First posted on arXiv 9 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 14; 2022: 13; 2023: 20; 2024: 9; 2025: 2; 2026: 1.

59 papers.

Sort:

By area:

  1. 2021Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Modelscited by 76
  2. 2021Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Timecited by 62
  3. 2023An Over-parameterized Exponential Regressioncited by 45
  4. 2021On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widthscited by 44
  5. 2022Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterizationcited by 36
  6. 2021Early-stopped neural networks are consistentcited by 34
  7. 2021Fast Sketching of Polynomial Kernels of Polynomial Degreecited by 33
  8. 2021Does Preprocessing Help Training Over-parameterized Neural Networks?cited by 32
  9. 2022Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessingcited by 29
  10. 2021Subquadratic Overparameterization for Shallow Neural Networkscited by 26
  11. 2023How to Protect Copyright Data in Optimization of Large Language Models?cited by 26
  12. 2024Evaluating the design space of diffusion-based generative modelscited by 26
38 more
  1. 2022Neural Tangent Kernel: A Surveycited by 23
  2. 2022Training Overparametrized Neural Networks in Sublinear Timecited by 22
  3. 2022A Sublinear Adversarial Training Algorithmcited by 22
  4. 2021Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networkscited by 21
  5. 2023Six Lectures on Linearized Neural Networkscited by 20
  6. 2024A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Modelscited by 20
  7. 2022Federated Adversarial Learning: A Framework with Convergence Analysiscited by 17
  8. 2022Bounding the Width of Neural Networks via Coupled Initialization -- A Worst Case Analysiscited by 15
  9. 2023Why Softmax Attention Outperforms Linear Attentioncited by 15
  10. 2024Exploring the Frontiers of Softmax: Provable Optimization, Applications in Diffusion Model, and Beyondcited by 15
  11. 2024Differential Privacy Mechanisms in Neural Tangent Kernel Regressioncited by 14
  12. 2021FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Convergence Analysiscited by 13
  13. 2023Unmasking Transformers: A Theoretical Approach to Data Recovery via Attention Weightscited by 12
  14. 2021On the Convergence of Shallow Neural Network Training with Randomly Masked Neuronscited by 11
  15. 2023Efficient SGD Neural Network Training via Sublinear Activated Neuron Identificationcited by 11
  16. 2021GIST: Distributed Training for Large-Scale Graph Convolutional Networkscited by 10
  17. 2023How many Neurons do we need? A refined Analysis for Shallow Networks trained with Gradient Descentcited by 10
  18. 2024Towards Infinite-Long Prefix in Transformercited by 10
  19. 2022Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networkscited by 9
  20. 2023Controlling the Inductive Bias of Wide Neural Networks by Modifying the Kernel's Spectrumcited by 9
  21. 2023An Automatic Learning Rate Schedule Algorithm for Achieving Faster Convergence and Steeper Descentcited by 9
  22. 2021Provable Convergence of Nesterov's Accelerated Gradient Method for Over-Parameterized Neural Networkscited by 7
  23. 2022Approximation results for Gradient Descent trained Shallow Neural Networks in 1dcited by 7
  24. 2023Convergence of Two-Layer Regression with Nonlinear Unitscited by 6
  25. 2023A Theoretical Insight into Attack and Defense of Gradient Leakage in Transformercited by 6
  26. 2022Finite Sample Identification of Wide Shallow Neural Networks with Biasescited by 5
  27. 2023Streaming Kernel PCA Algorithm With Small Spacecited by 5
  28. 2023Structured and Fast Optimization: The Kronecker SGD Algorithmcited by 5
  29. 2023Memory capacity of two layer neural networks with smooth activationscited by 5
  30. 2021A Convergence Theory Towards Practical Over-parameterized Deep Neural Networkscited by 4
  31. 2023Query Complexity of Active Learning for Function Family With Nearly Orthogonal Basiscited by 4
  32. 2023Understanding Deep Neural Networks via Linear Separability of Hidden Layerscited by 4
  33. 2024Unlocking the Theory Behind Scaling 1-Bit Neural Networkscited by 4
  34. 2023Pruning Before Training May Improve Generalization, Provablycited by 3
  35. 2024Analyzing the Neural Tangent Kernel of Periodically Activated Coordinate Networkscited by 3
  36. 2024Convergence of Implicit Gradient Descent for Training Two-Layer Physics-Informed Neural Networkscited by 3
  37. 2021Fast Graph Neural Tangent Kernel via Kronecker Sketchingcited by 2
  38. 2023Neural Networks with Sparse Activation Induced by Large Bias: Tighter Analysis with Bias-Generalized NTKcited by 2

50 most cited of 59 All 59 (JSON)