Geometric Dyson Brownian Motions and the Free Log-Normal Limit for a Non-Square Gaussian Matrix Product
Not on the map. Why?
Abstract (arXiv)
We study the squared singular value spectrum of a non-square product of independent real Gaussian matrices, equivalently the feature covariance spectrum of a deep linear neural network at initialization. Starting from the fixed-m covariance diffusion previously obtained in the proportional depth-width limit, we record an equivalent matrix realization, describe its affine invariance, and derive the interacting diffusion satisfied by its eigenvalues. We then take a second limit, sending m→∞ on the accelerated spectral clock τ=mt, which corresponds in this sequential construction to the relation dm/n→barτ. We establish convergence of the empirical spectral measure path to a deterministic mean-field limit and derive a closed Burgers equation for its T-transform. Together with the proportional depth-width limit, these results give a rigorous sequential route from the deep non-square Gaussian product to the free log-normal limit of its feature covariance spectrum; for more general initial laws, the transform yields a free multiplicative convolution form. We further analyze the support of the free log-normal law, give a fixed point iteration for numerical evaluation and a formal Marchenko--Pastur approximation at small time, and use the limiting spectrum to predict the risk in a toy random feature model.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 47 of its 91 references that are in the map. First posted on arXiv 29 Jun 2026.
Builds on
47 of 91 references.
References · 47
- 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
- 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 738
- 2018On Lazy Training in Differentiable Programmingcited by 675
- 2017Deep Neural Networks as Gaussian Processescited by 622
- 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 574
- 2019On Exact Computation with an Infinitely Wide Neural Netcited by 573
- 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 466
- 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 426
- 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 413
- 2022Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfercited by 288
- 2021Deep learning: a statistical viewpointcited by 240
- 2020Feature Learning in Infinite-Width Neural Networkscited by 190
35 more
- 2019Linearized two-layers neural networks in high dimensioncited by 155
- 2019Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivationcited by 149
- 2020Tensor Programs II: Neural Tangent Kernel for Any Architecturecited by 143
- 2019Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow ReLU networkscited by 121
- 2019Finite Depth and Width Corrections to the Neural Tangent Kernelcited by 115
- 2022Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networkscited by 105
- 2023Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networkscited by 97
- 2019How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?cited by 84
- 2017A Random Matrix Approach to Neural Networkscited by 80
- 2020Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networkscited by 77
- 2023Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limitcited by 66
- 2018Products of Many Large Random Matrices and Gradients in Deep Neural Networkscited by 49
- 2020Stable ResNetcited by 49
- 2023The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limitcited by 47
- 2022The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initializationcited by 41
- 2023Deterministic equivalent and error universality of deep random features learningcited by 35
- 2021The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initializationcited by 33
- 2022Neural Tangent Kernel Beyond the Infinite-Width Limit: Effects of Depth and Initializationcited by 30
- 2024Nonlinear spiked covariance matrices and signal propagation in deep neural networkscited by 26
- 2022Quantitative Gaussian Approximation of Randomly Initialized Deep Neural Networkscited by 25
- 2023Precise Asymptotic Analysis of Deep Random Feature Modelscited by 22
- 2022Bayesian Interpolation with Deep Linear Networkscited by 21
- 2023Width and Depth Limits Commute in Residual Networkscited by 21
- 2023Quantitative CLTs in Deep Neural Networkscited by 20
- 2023Wide Deep Neural Networks with Gaussian Weights are Very Close to Gaussian Processescited by 13
- 2024Universality of Kernel Random Matrices and Kernel Regression in the Quadratic Regimecited by 12
- 2023Non-asymptotic approximations of Gaussian neural networks via second-order Poincar'e inequalitiescited by 11
- 2023Gaussian random field approximation via Stein's method with applications to wide random neural networkscited by 11
- 2025Entropic bounds for conditionally Gaussian vectors and applications to neural networkscited by 7
- 2024Bayesian Inference with Deep Weakly Nonlinear Networkscited by 5
- 2025Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weightscited by 5
- 2023Differential Equation Scaling Limits of Shaped and Unshaped Neural Networkscited by 4
- 2024Proportional infinite-width infinite-depth limit for deep linear neural networkscited by 3
- 2026Universality in Deep Neural Networks: An approach via the Lindeberg exchange principlecited by 2
- 2026Bayesian Inference with Shaped Deep Non-linear MLPscited by 1
Built on by
Not cited yet.