# Grokking as Dimensional Phase Transition in Neural Networks

arXiv: https://arxiv.org/abs/2604.04655 · HTML: https://mlsanity.com/paper/2604.04655/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 2 references that are in the map.
- First posted on arXiv 6 Apr 2026.

## Built on by (most cited first, 4 of 4)

- [Weight Decay Regimes in Grokking Transformers: Cheap Online Diagnostics](https://mlsanity.com/paper/2605.20441/) (2026)
- [What Does the Weight Norm Control in Grokking? Logit-Scale Mediation under Cross-Entropy](https://mlsanity.com/paper/2606.18465/) (2026)
- [At-Grok Is Not Converged:A Measurement-Validity Audit for Grokking Representation Metrics](https://mlsanity.com/paper/2607.06639/) (2026)
- [Finite-Size Gradient Transport in Large Language Model Pretraining: From Cascade Size to Intensive Transport Efficiency](https://mlsanity.com/paper/2605.02968/) (2026)

## Abstract (arXiv)

Neural network grokking -- the abrupt memorization-to-generalization transition -- challenges our understanding of learning dynamics. Through finite-size scaling of gradient avalanche dynamics across eight model scales, we find that grokking is a textitdimensional phase transition: effective dimensionality~D crosses from sub-diffusive (subcritical, D < 1) to super-diffusive (supercritical, D > 1) at generalization onset, exhibiting self-organized criticality (SOC). Crucially, D reflects gradient field geometry, not network architecture: synthetic i.i.d. Gaussian gradients maintain D ≈ 1 regardless of graph topology, while real training exhibits dimensional excess from backpropagation correlations. The grokking-localized D(t) crossing -- robust across topologies -- offers new insight into the trainability of overparameterized networks.