# Deep Networks Always Grok and Here is Why

arXiv: https://arxiv.org/abs/2402.15555 · HTML: https://mlsanity.com/paper/2402.15555/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 41 references that are in the map.
- First posted on arXiv 23 Feb 2024.

## Builds on (most cited first, 29 of 29)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Progress measures for grokking via mechanistic interpretability](https://mlsanity.com/paper/2301.05217/) (2023)
- [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020)
- [Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets](https://mlsanity.com/paper/2201.02177/) (2022)
- [On the Number of Linear Regions of Deep Neural Networks](https://mlsanity.com/paper/1402.1869/) (2014)
- [Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks](https://mlsanity.com/paper/1703.02930/) (2017)
- [On the Expressive Power of Deep Neural Networks](https://mlsanity.com/paper/1606.05336/) (2016)
- [Sensitivity and Generalization in Neural Networks: an Empirical Study](https://mlsanity.com/paper/1802.08760/) (2018)
- [Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit](https://mlsanity.com/paper/2207.08799/) (2022)
- [Complexity of Linear Regions in Deep Networks](https://mlsanity.com/paper/1901.09021/) (2019)
- [Adversarial Robustness through Local Linearization](https://mlsanity.com/paper/1907.02610/) (2019)
- [Towards Understanding Sharpness-Aware Minimization](https://mlsanity.com/paper/2206.06232/) (2022)
- [Explaining grokking through circuit efficiency](https://mlsanity.com/paper/2309.02390/) (2023)
- [Benign Overfitting and Grokking in ReLU Networks for XOR Cluster Data](https://mlsanity.com/paper/2310.02541/) (2023)
- [SplineCam: Exact Visualization and Characterization of Deep Network Geometry and Decision Boundaries](https://mlsanity.com/paper/2302.12828/) (2023)
- [Why Robust Generalization in Deep Learning is Difficult: Perspective of Expressive Power](https://mlsanity.com/paper/2205.13863/) (2022)
- [Polarity Sampling: Quality and Diversity Control of Pre-Trained Generative Networks via Singular Values](https://mlsanity.com/paper/2203.01993/) (2022)
- [POLICE: Provably Optimal Linear Constraint Enforcement for Deep Neural Networks](https://mlsanity.com/paper/2211.01340/) (2022)
- [A Blessing of Dimensionality in Membership Inference through Regularization](https://mlsanity.com/paper/2205.14055/) (2022)
- [Are All Linear Regions Created Equal?](https://mlsanity.com/paper/2202.11749/) (2022)
- [Implicit Regularization in Over-parameterized Neural Networks](https://mlsanity.com/paper/1903.01997/) (2019)
- [Test Sample Accuracy Scales with Training Sample Density in Neural Networks](https://mlsanity.com/paper/2106.08365/) (2021)
- [Batch Normalization Explained](https://mlsanity.com/paper/2209.14778/) (2022)
- [Max-Affine Spline Insights Into Deep Network Pruning](https://mlsanity.com/paper/2101.02338/) (2021)
- [Towards Deep Learning Models Resistant to Transfer-based Adversarial Attacks via Data-centric Robust Learning](https://mlsanity.com/paper/2310.09891/) (2023)

## Built on by (most cited first, 30 of 51)

- [Grokking at the Edge of Numerical Stability](https://mlsanity.com/paper/2501.04697/) (2025)
- [Approaching Deep Learning through the Spectral Dynamics of Weights](https://mlsanity.com/paper/2408.11804/) (2024)
- [Understanding Visual Feature Reliance through the Lens of Complexity](https://mlsanity.com/paper/2407.06076/) (2024)
- [Let Me Grok for You: Accelerating Grokking via Embedding Transfer from a Weaker Model](https://mlsanity.com/paper/2504.13292/) (2025)
- [On the Ability of Deep Networks to Learn Symmetries from Data: A Neural Kernel Theory](https://mlsanity.com/paper/2412.11521/) (2024)
- [Grokking in LLM Pretraining? Monitor Memorization-to-Generalization without Test](https://mlsanity.com/paper/2506.21551/) (2025)
- [On the Local Complexity of Linear Regions in Deep ReLU Networks](https://mlsanity.com/paper/2412.18283/) (2024)
- [Position: Solve Layerwise Linear Models First to Understand Neural Dynamical Phenomena (Neural Collapse, Emergence, Lazy/Rich Regime, and Grokking)](https://mlsanity.com/paper/2502.21009/) (2025)
- [The Geometry of Grokking: Norm Minimization on the Zero-Loss Manifold](https://mlsanity.com/paper/2511.01938/) (2025)
- [Progress Measures for Grokking on Real-world Tasks](https://mlsanity.com/paper/2405.12755/) (2024)
- [Flatness is Necessary, Neural Collapse is Not: Rethinking Generalization via Grokking](https://mlsanity.com/paper/2509.17738/) (2025)
- [To Grok Grokking: Provable Grokking in Ridge Regression](https://mlsanity.com/paper/2601.19791/) (2026)
- [What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Models](https://mlsanity.com/paper/2408.08307/) (2024)
- [Mechanistic Insights into Grokking from the Embedding Layer](https://mlsanity.com/paper/2505.15624/) (2025)
- [Quantum Hamiltonian Embedding of Images for Data Reuploading Classifiers](https://mlsanity.com/paper/2407.14055/) (2024)
- [Time to Spike? Understanding the Representational Power of Spiking Neural Networks in Discrete Time](https://mlsanity.com/paper/2505.18023/) (2025)
- [Learning words in groups: fusion algebras, tensor ranks and grokking](https://mlsanity.com/paper/2509.06931/) (2025)
- [Explaining Grokking and Information Bottleneck through Neural Collapse Emergence](https://mlsanity.com/paper/2509.20829/) (2025)
- [Is your algorithm unlearning or untraining?](https://mlsanity.com/paper/2604.07962/) (2026)
- [Model Capacity Determines Grokking through Competing Memorisation and Generalisation Speeds](https://mlsanity.com/paper/2605.09724/) (2026)
- [Acceleration of Grokking in Learning Arithmetic Operations via Kolmogorov-Arnold Representation](https://mlsanity.com/paper/2405.16658/) (2024)
- [Constraining the outputs of ReLU neural networks](https://mlsanity.com/paper/2508.03867/) (2025)
- [Grokking as a Falsifiable Finite-Size Transition](https://mlsanity.com/paper/2603.24746/) (2026)
- [The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts](https://mlsanity.com/paper/2604.11962/) (2026)
- [Grokking vs. Learning: Same Features, Different Encodings](https://mlsanity.com/paper/2502.01739/) (2025)
- [NeuralGrok: Accelerate Grokking by Neural Gradient Transformation](https://mlsanity.com/paper/2504.17243/) (2025)
- [GrokAlign: Geometric Characterisation and Acceleration of Grokking](https://mlsanity.com/paper/2506.12284/) (2025)
- [Topological Signatures of ReLU Neural Network Activation Patterns](https://mlsanity.com/paper/2510.12700/) (2025)
- [Topological Signatures of Grokking](https://mlsanity.com/paper/2605.06352/) (2026)
- [Two Speeds of Learning: A Representation-Readout Decomposition of Grokking and Double Descent](https://mlsanity.com/paper/2605.27078/) (2026)

## Abstract (arXiv)

Grokking, or delayed generalization, is a phenomenon where generalization in a deep neural network (DNN) occurs long after achieving near zero training error. Previous studies have reported the occurrence of grokking in specific controlled settings, such as DNNs initialized with large-norm parameters or transformers trained on algorithmic datasets. We demonstrate that grokking is actually much more widespread and materializes in a wide range of practical settings, such as training of a convolutional neural network (CNN) on CIFAR10 or a Resnet on Imagenette. We introduce the new concept of delayed robustness, whereby a DNN groks adversarial examples and becomes robust, long after interpolation and/or generalization. We develop an analytical explanation for the emergence of both delayed generalization and delayed robustness based on the local complexity of a DNN's input-output mapping. Our local complexity measures the density of so-called linear regions (aka, spline partition regions) that tile the DNN input space and serves as a utile progress measure for training. We provide the first evidence that, for classification problems, the linear regions undergo a phase transition during training whereafter they migrate away from the training samples (making the DNN mapping smoother there) and towards the decision boundary (making the DNN mapping less smooth there). Grokking occurs post phase transition as a robust partition of the input space thanks to the linearization of the DNN mapping around the training points. Website: https://bit.ly/grok-adversarial