mlsanityData to 14 Oct 2026

A Tale of Two Circuits: Grokking as Competition of Sparse and Dense Subnetworks

arXiv 2303.11873 ↗First posted 21 Mar 2023cs.LGcited by 83

Not on the map. Why?

Abstract (arXiv)

Grokking is a phenomenon where a model trained on an algorithmic task first overfits but, then, after a large amount of additional training, undergoes a phase transition to generalize perfectly. We empirically study the internal structure of networks undergoing grokking on the sparse parity task, and find that the grokking phase transition corresponds to the emergence of a sparse subnetwork that dominates model predictions. On an optimization level, we find that this subnetwork arises when a small subset of neurons undergoes rapid norm growth, whereas the other neurons in the network decay slowly in norm. Thus, we suggest that the grokking phase transition can be understood to emerge from competition of two largely distinct subnetworks: a dense one that dominates before the transition and generalizes poorly, and a sparse one that dominates afterwards.

Key facts. Cited by 83 papers in the map (data through 14 Oct 2026). Builds on 7 of its 11 references that are in the map. First posted on arXiv 21 Mar 2023.

Builds on

7 of 11 references.

References · 7

  1. 2022Emergent Abilities of Large Language Modelscited by 2,512
  2. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  3. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 629
  4. 2022Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limitcited by 187
  5. 2022Omnigrok: Grokking Beyond Algorithmic Datacited by 146
  6. 2022The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenoncited by 94
  7. 2020Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descentcited by 40

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 18; 2024: 22; 2025: 15; 2026: 28.

83 papers.

Sort:

By area:

  1. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
  2. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  3. 2023Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMscited by 124
  4. 2024The Evolution of Statistical Induction Heads: In-Context Learning Markov Chainscited by 118
  5. 2023Explaining grokking through circuit efficiencycited by 111
  6. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 85
  7. 2023Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokkingcited by 80
  8. 2024Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalizationcited by 79
  9. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 51
  10. 2025Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biasescited by 37
  11. 2023Benign Overfitting and Grokking in ReLU Networks for XOR Cluster Datacited by 33
  12. 2024Approaching Deep Learning through the Spectral Dynamics of Weightscited by 31
38 more
  1. 2024A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Languagecited by 26
  2. 2023Grokking in Linear Estimators -- A Solvable Model that Groks without Understandingcited by 25
  3. 2023Interpretability Illusions in the Generalization of Simplified Modelscited by 25
  4. 2024Mechanistic?cited by 25
  5. 2023To grok or not to grok: Disentangling generalization and memorization on corrupted algorithmic datasetscited by 24
  6. 2023Grokking Group Multiplication with Cosetscited by 24
  7. 2024Learning Syntax Without Planting Trees: Understanding Hierarchical Generalization in Transformerscited by 24
  8. 2024Towards Uncovering How Large Language Model Works: An Explainability Perspectivecited by 22
  9. 2025How Do Transformers Learn Variable Binding in Symbolic Programs?cited by 21
  10. 2024Grokking Modular Polynomialscited by 18
  11. 2024Information-Theoretic Progress Measures reveal Grokking is an Emergent Phase Transitioncited by 17
  12. 2023Latent State Models of Training Dynamicscited by 16
  13. 2025Provable Scaling Laws of Feature Emergence from Learning Dynamics of Grokkingcited by 15
  14. 2023Grokking as Compression: A Nonlinear Complexity Perspectivecited by 14
  15. 2024Towards Empirical Interpretation of Internal Circuits and Properties in Grokked Transformers on Modular Polynomialscited by 14
  16. 2023Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luckcited by 13
  17. 2023Grokking Beyond Neural Networks: An Empirical Exploration with Model Complexitycited by 13
  18. 2023In-Context Learning Dynamics with Random Binary Sequencescited by 13
  19. 2023Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimizationcited by 13
  20. 2025Let Me Grok for You: Accelerating Grokking via Embedding Transfer from a Weaker Modelcited by 13
  21. 2023Understanding Grokking Through A Robustness Viewpointcited by 12
  22. 2024Complexity Matters: Dynamics of Feature Learning in the Presence of Spurious Correlationscited by 12
  23. 2024Grokking at the Edge of Linear Separabilitycited by 12
  24. 2026Low-Dimensional and Transversely Curved Optimization Dynamics in Grokkingcited by 11
  25. 2024The Heuristic Core: Understanding Subnetwork Generalization in Pretrained Language Modelscited by 10
  26. 2024Sometimes I am a Tree: Data Drives Unstable Hierarchical Generalizationcited by 10
  27. 2025Grokking in LLM Pretraining? Monitor Memorization-to-Generalization without Testcited by 10
  28. 2026The Geometry of Multi-Task Grokking: Transverse Instability, Superposition, and Weight Decay Phase Structurecited by 9
  29. 2024Progress Measures for Grokking on Real-world Taskscited by 8
  30. 2025Is Grokking a Computational Glass Relaxation?cited by 8
  31. 2025Grokking and Generalization Collapse: Insights from textttHTSR theorycited by 8
  32. 2026To Grok Grokking: Provable Grokking in Ridge Regressioncited by 8
  33. 2025Random Scaling of Emergent Capabilitiescited by 7
  34. 2025What Happens During the Loss Plateau? Understanding Abrupt Learning in Transformerscited by 7
  35. 2023Bridging Lottery Ticket and Grokking: Understanding Grokking from Inner Structure of Networkscited by 6
  36. 2025Emergent Symbol-like Number Variables in Artificial Neural Networkscited by 6
  37. 2026Spectral Edge Dynamics of Training Trajectories: Signal--Noise Geometry Across Scalescited by 6
  38. 2026A Systematic Empirical Study of Grokking: Depth, Architecture, Activation, and Regularizationcited by 6

50 most cited of 83 All 83 (JSON)