# Diffusion Operator Geometry of Feedforward Representations

arXiv: https://arxiv.org/abs/2605.01107 · HTML: https://mlsanity.com/paper/2605.01107/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 57 references that are in the map.
- First posted on arXiv 1 May 2026.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [Insights on representational similarity in neural networks with canonical correlation](https://mlsanity.com/paper/1806.05759/) (2018)
- [Intrinsic dimension of data representations in deep neural networks](https://mlsanity.com/paper/1905.12784/) (2019)
- [A Geometric Analysis of Neural Collapse with Unconstrained Features](https://mlsanity.com/paper/2105.02375/) (2021)
- [Deep Information Propagation](https://mlsanity.com/paper/1611.01232/) (2016)
- [A mathematical perspective on Transformers](https://mlsanity.com/paper/2312.10794/) (2023)
- [Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Path](https://mlsanity.com/paper/2106.02073/) (2021)
- [The emergence of clusters in self-attention dynamics](https://mlsanity.com/paper/2305.05465/) (2023)
- [Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity](https://mlsanity.com/paper/1602.05897/) (2016)
- [Topology of deep neural networks](https://mlsanity.com/paper/2004.06093/) (2020)
- [Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks](https://mlsanity.com/paper/2205.09653/) (2022)
- [Community Detection on Networks with Ricci Flow](https://mlsanity.com/paper/1907.03993/) (2019)
- [Clustering in Causal Attention Masking](https://mlsanity.com/paper/2411.04990/) (2024)
- [The Mean-Field Dynamics of Transformers](https://mlsanity.com/paper/2512.01868/) (2025)
- [Lipschitz regularity of graph Laplacians on random data clouds](https://mlsanity.com/paper/2007.06679/) (2020)
- [Assessing Neural Network Representations During Training Using Noise-Resilient Diffusion Spectral Entropy](https://mlsanity.com/paper/2312.04823/) (2023)
- [Ollivier-Ricci curvature convergence in random geometric graphs](https://mlsanity.com/paper/2008.01209/) (2020)
- [Computing Diffusion Geometry](https://mlsanity.com/paper/2602.06006/) (2026)
- [Continuum Limits of Ollivier's Ricci Curvature on data clouds: pointwise consistency and global lower bounds](https://mlsanity.com/paper/2307.02378/) (2023)
- [Deep Learning as Ricci Flow](https://mlsanity.com/paper/2404.14265/) (2024)
- [Exploring the Manifold of Neural Networks Using Diffusion Geometry](https://mlsanity.com/paper/2411.12626/) (2024)
- [Neural Feature Geometry Evolves as Discrete Ricci Flow](https://mlsanity.com/paper/2509.22362/) (2025)

## Built on by (most cited first, 1 of 1)

- [Auditing Finite-Lag Statistics of Recurrent Representations](https://mlsanity.com/paper/2607.01746/) (2026)

## Abstract (arXiv)

Feedforward neural networks transform data through learned representations whose geometry shapes how classes separate and relate across successive layers. We study that geometry through diffusion operators. Each feature-cloud snapshot is assigned a Gaussian-kernel Markov operator, giving a smooth description of one-step transport between classes from which spectral, boundary, and local geometric information can be read. We define the empirical class chain, state the condition under which it is an exact Markov quotient, and derive both the corresponding population transition and a simpler overlap chain based on expected class affinities. For balanced shared-covariance Gaussian class-conditional snapshots these affinities have closed forms controlled by a regularized Mahalanobis separation, which yields explicit expressions for leakage and coarse spectral behaviour. We further show that operator observables vary smoothly under feature perturbations, whereas hard neighborhood graphs are controlled by neighbor-order margins. Experiments on CIFAR-10 and CIFAR-100 ResNet-18 representations find that class transport becomes increasingly persistent with depth while retaining structured relations between classes, and that the diffusion class chain is more stable than its k-nearest-neighbor counterpart under matched perturbations.