mlsanityData to 14 Oct 2026

How to explain grokking

arXiv 2412.18624 ↗First posted 17 Dec 2024cond-mat.stat-mech cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Explanation of grokking (delayed generalization) in learning is given by modeling grokking by the stochastic gradient Langevin dynamics (Brownian motion) and applying the ideas of thermodynamics.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 6 of its 15 references that are in the map. First posted on arXiv 17 Dec 2024.

Builds on

6 of 15 references.

References · 6

  1. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  2. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 629
  3. 2020Loss landscapes and optimization in over-parameterized non-linear systems and neural networkscited by 233
  4. 2022Towards Understanding Grokking: An Effective Theory of Representation Learningcited by 230
  5. 2022Omnigrok: Grokking Beyond Algorithmic Datacited by 146
  6. 2024Control of Overfitting with Physicscited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Can Stationary Distributions of Scale-Invariant Neural Networks Be Described by the Thermodynamics of an Ideal Gas?cited by 0