mlsanityData to 7 Oct 2026

A Gaussian Comparison Theorem for Training Dynamics in Machine Learning

arXiv 2603.09310 ↗First posted 10 Mar 2026cs.LG math.PR stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

We study training algorithms with data following a Gaussian mixture model. For a specific family of such algorithms, we present a non-asymptotic result, connecting the evolution of the model to a surrogate dynamical system, which can be easier to analyze. The proof of our result is based on the celebrated Gordon comparison theorem. Using our theorem, we rigorously prove the validity of the dynamic mean-field (DMF) expressions in the asymptotic scenarios. Moreover, we suggest an iterative refinement scheme to obtain more accurate expressions in non-asymptotic scenarios. We specialize our theory to the analysis of training a perceptron model with a generic first-order (full-batch) algorithm and demonstrate that fluctuation parameters in a non-asymptotic domain emerge in addition to the DMF kernels.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 37 references that are in the map. First posted on arXiv 10 Mar 2026.

Builds on

22 of 37 references.

References · 22

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 748
  3. 2019On Exact Computation with an Infinitely Wide Neural Netcited by 572
  4. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
  5. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 463
  6. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 422
  7. 2017High-dimensional dynamics of generalization error in neural networkscited by 238
  8. 2019Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networkscited by 221
  9. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 218
  10. 2019Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limitcited by 187
  11. 2019Towards moderate overparameterization: global convergence guarantees for training shallow neural networkscited by 176
  12. 2017Theoretical insights into the optimization landscape of over-parameterized shallow neural networkscited by 128
10 more
  1. 2019Finite Depth and Width Corrections to the Neural Tangent Kernelcited by 114
  2. 2016A Variational Perspective on Accelerated Methods in Optimizationcited by 108
  3. 2022Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networkscited by 104
  4. 2019What Can ResNet Learn Efficiently, Going Beyond Kernels?cited by 91
  5. 2022High-dimensional limit theorems for SGD: Effective dynamics and critical scalingcited by 69
  6. 2020Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classificationcited by 60
  7. 2023Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networkscited by 42
  8. 2023Precise Asymptotic Analysis of Deep Random Feature Modelscited by 22
  9. 2022Neural Tangent Kernel Analysis of Deep Narrow Neural Networkscited by 11
  10. 2026Disordered Dynamics in High Dimensions: Connections to Random Matrices and Machine Learningcited by 4

Built on by

Not cited yet.