# Gaussian Universality in Neural Network Dynamics with Generalized Structured Input Distributions

arXiv: https://arxiv.org/abs/2405.00642 · HTML: https://mlsanity.com/paper/2405.00642/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 23 references that are in the map.
- First posted on arXiv 1 May 2024.

## Builds on (most cited first, 3 of 3)

- [The generalization error of random features regression: Precise asymptotics and double descent curve](https://mlsanity.com/paper/1908.05355/) (2019)
- [How Two-Layer Neural Networks Learn, One (Giant) Step at a Time](https://mlsanity.com/paper/2305.18270/) (2023)
- [Mean-field inference methods for neural networks](https://mlsanity.com/paper/1911.00890/) (2019)

## Built on by (most cited first, 1 of 1)

- [Dataset Complexity Shapes Finite-Distance Loss Geometry in Neural Networks](https://mlsanity.com/paper/2608.22361/) (2026)

## Abstract (arXiv)

Analyzing neural network dynamics via stochastic gradient descent (SGD) is crucial to building theoretical foundations for deep learning. Previous work has analyzed structured inputs within the textithidden manifold model, often under the simplifying assumption of a Gaussian distribution. We extend this framework by modeling inputs as Gaussian mixtures to better represent complex, real-world data. Through empirical and theoretical investigation, we demonstrate that with proper standardization, the learning dynamics converges to the behavior seen in the simple Gaussian case. This finding exhibits a form of universality, where diverse structured distributions yield results consistent with Gaussian assumptions, thereby strengthening the theoretical understanding of deep learning models.