mlsanityData to 14 Oct 2026

The Dynamics of Learning: A Random Matrix Approach

arXiv 1805.11917 ↗First posted 30 May 2018stat.ML cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Understanding the learning dynamics of neural networks is one of the key issues for the improvement of optimization algorithms as well as for the theoretical comprehension of why deep neural nets work so well today. In this paper, we introduce a random matrix-based framework to analyze the learning dynamics of a single-layer linear network on a binary classification problem, for data of simultaneously large dimension and size, trained by gradient descent. Our results provide rich insights into common questions in neural nets, such as overfitting, early stopping and the initialization of training, thereby opening the door for future studies of more elaborate structures and models appearing in today's neural networks.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 2; 2023: 1; 2024: 5; 2025: 2; 2026: 1.

15 papers.

Sort:

By area:

  1. 2022Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensionscited by 19
  2. 2021Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature modelcited by 17
  3. 2023Over-training with Mixup May Hurt Generalizationcited by 13
  4. 2021A Random Matrix Perspective on Random Tensorscited by 12
  5. 2024SPQR: Controlling Q-ensemble Independence with Spiked Random Model for Reinforcement Learningcited by 8
  6. 2021Two-way kernel matrix puncturing: towards resource-efficient PCA and spectral clusteringcited by 7
  7. 2021Random matrices in service of ML footprint: ternary random features with no performance losscited by 7
  8. 2024Deep Equilibrium Models are Almost Equivalent to Not-so-deep Explicit Models for High-dimensional Gaussian Mixturescited by 6
  9. 2024High-dimensional analysis of ridge regression for non-identically distributed data with a variance profilecited by 5
  10. 2025Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Modelscited by 4
  11. 2024"Lossless" Compression of Deep Neural Networks: A High-dimensional Neural Tangent Kernel Approachcited by 3
  12. 2022Deep Regression Ensemblescited by 1
3 more
  1. 2025A Random Matrix Theory Perspective on the Learning Dynamics of Multi-head Latent Attentioncited by 1
  2. 2024SnatchML: Hijacking ML models without Training Accesscited by 0
  3. 2026A Data-dependent Early Stopping Rule using Rademacher Complexity with L1-normcited by 0