mlsanityData to 14 Oct 2026

Grokking as a Falsifiable Finite-Size Transition

arXiv 2603.24746 ↗First posted 25 Mar 2026cs.LG cond-mat.stat-mech cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Grokking -- the delayed onset of generalization after early memorization -- is often described with phase-transition language, but that claim has lacked falsifiable finite-size inputs. Here we supply those inputs by treating the group order p of mathbbZp as an admissible extensive variable and a held-out spectral head-tail contrast as a representation-level order parameter, then apply a condensed-matter-style diagnostic chain to coarse-grid sweeps and a dense near-critical addition audit. Binder-like crossings reveal a shared finite-size boundary, and susceptibility comparison strongly disfavors a smooth-crossover interpretation (DeltaAIC=16.8 in the near-critical audit). Phase-transition language in grokking can therefore be tested as a quantitative finite-size claim rather than invoked as analogy alone, although the transition order remains unresolved at present.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 29 of its 51 references that are in the map. First posted on arXiv 25 Mar 2026.

Builds on

29 of 51 references.

References · 29

  1. 2020Scaling Laws for Neural Language Modelscited by 6,586
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  3. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  4. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  5. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  6. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
  7. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 629
  8. 2021Explaining Neural Scaling Lawscited by 363
  9. 2018Essentially No Barriers in Neural Network Energy Landscapecited by 330
  10. 2019Intrinsic dimension of data representations in deep neural networkscited by 282
  11. 2022Towards Understanding Grokking: An Effective Theory of Representation Learningcited by 230
  12. 2018Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learningcited by 215
17 more
  1. 2022Omnigrok: Grokking Beyond Algorithmic Datacited by 146
  2. 2023A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operationscited by 144
  3. 2023A Tale of Two Circuits: Grokking as Competition of Sparse and Dense Subnetworkscited by 83
  4. 2023Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokkingcited by 80
  5. 2019Large Scale Structure of Neural Network Loss Landscapescited by 68
  6. 2024Deep Networks Always Grok and Here is Whycited by 51
  7. 2023Grokking as a First Order Phase Transition in Two Layer Networkscited by 48
  8. 2025Grokking at the Edge of Numerical Stabilitycited by 46
  9. 2024Grokfast: Accelerated Grokking by Amplifying Slow Gradientscited by 41
  10. 2023Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grokcited by 35
  11. 2022Grokking phase transitions in learning local rules with gradient descentcited by 28
  12. 2024The Complexity Dynamics of Grokkingcited by 22
  13. 2024Information-Theoretic Progress Measures reveal Grokking is an Emergent Phase Transitioncited by 17
  14. 2015On Simplicity and Complexity in the Brave New World of Large-Scale Neurosciencecited by 15
  15. 2025Grokking Beyond the Euclidean Norm of Model Parameterscited by 15
  16. 2022Statistical mechanics of continual learning: variational principle and mean-field potentialcited by 9
  17. 2025Is Grokking a Computational Glass Relaxation?cited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Weight Decay Regimes in Grokking Transformers: Cheap Online Diagnosticscited by 2
  2. 2026What Does the Weight Norm Control in Grokking? Logit-Scale Mediation under Cross-Entropycited by 1
  3. 2026Phase Transitions in Driven Informational Systems: A Two-Field Perspective on Learning Theory and Non-Equilibrium Chemistrycited by 0