# To grok or not to grok: Disentangling generalization and memorization on corrupted algorithmic datasets

arXiv: https://arxiv.org/abs/2310.13061 · HTML: https://mlsanity.com/paper/2310.13061/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 37 references that are in the map.
- First posted on arXiv 19 Oct 2023.

## Builds on (most cited first, 26 of 26)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Progress measures for grokking via mechanistic interpretability](https://mlsanity.com/paper/2301.05217/) (2023)
- [Quantifying Memorization Across Neural Language Models](https://mlsanity.com/paper/2202.07646/) (2022)
- [Exploring Generalization in Deep Learning](https://mlsanity.com/paper/1706.08947/) (2017)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [Deep Learning on a Data Diet: Finding Important Examples Early in Training](https://mlsanity.com/paper/2107.07075/) (2021)
- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [Are we done with ImageNet?](https://mlsanity.com/paper/2006.07159/) (2020)
- [Not All Samples Are Created Equal: Deep Learning with Importance Sampling](https://mlsanity.com/paper/1803.00942/) (2018)
- [Norm-Based Capacity Control in Neural Networks](https://mlsanity.com/paper/1503.00036/) (2015)
- [Towards Understanding Grokking: An Effective Theory of Representation Learning](https://mlsanity.com/paper/2205.10343/) (2022)
- [Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models](https://mlsanity.com/paper/2205.10770/) (2022)
- [Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit](https://mlsanity.com/paper/2207.08799/) (2022)
- [The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks](https://mlsanity.com/paper/2306.17844/) (2023)
- [Scaling Laws and Interpretability of Learning from Repeated Data](https://mlsanity.com/paper/2205.10487/) (2022)
- [Omnigrok: Grokking Beyond Algorithmic Data](https://mlsanity.com/paper/2210.01117/) (2022)
- [On the importance of single directions for generalization](https://mlsanity.com/paper/1803.06959/) (2018)
- [Dropout Training as Adaptive Regularization](https://mlsanity.com/paper/1307.1493/) (2013)
- [Masking: A New Perspective of Noisy Supervision](https://mlsanity.com/paper/1805.08193/) (2018)
- [A Tale of Two Circuits: Grokking as Competition of Sparse and Dense Subnetworks](https://mlsanity.com/paper/2303.11873/) (2023)
- [Deep Learning from Noisy Image Labels with Quality Embedding](https://mlsanity.com/paper/1711.00583/) (2017)
- [Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grok](https://mlsanity.com/paper/2306.13253/) (2023)
- [Grokking phase transitions in learning local rules with gradient descent](https://mlsanity.com/paper/2210.15435/) (2022)
- [Distinct types of eigenvector localization in networks](https://mlsanity.com/paper/1505.06024/) (2015)

## Built on by (most cited first, 24 of 24)

- [Learning to grok: Emergence of in-context learning and skill composition in modular arithmetic tasks](https://mlsanity.com/paper/2406.02550/) (2024)
- [Critical Data Size of Language Models from a Grokking Perspective](https://mlsanity.com/paper/2401.10463/) (2024)
- [Towards Uncovering How Large Language Model Works: An Explainability Perspective](https://mlsanity.com/paper/2402.10688/) (2024)
- [Information-Theoretic Progress Measures reveal Grokking is an Emergent Phase Transition](https://mlsanity.com/paper/2408.08944/) (2024)
- [FIRE-Bench: Evaluating AI Agents on the Rediscovery of Scientific Insights](https://mlsanity.com/paper/2602.02905/) (2026)
- [Provable Scaling Laws of Feature Emergence from Learning Dynamics of Grokking](https://mlsanity.com/paper/2509.21519/) (2025)
- [Towards Empirical Interpretation of Internal Circuits and Properties in Grokked Transformers on Modular Polynomials](https://mlsanity.com/paper/2402.16726/) (2024)
- [Can Interpretation Predict Behavior on Unseen Data?](https://mlsanity.com/paper/2507.06445/) (2025)
- [Learning words in groups: fusion algebras, tensor ranks and grokking](https://mlsanity.com/paper/2509.06931/) (2025)
- [On the geometry and topology of representations: the manifolds of modular addition](https://mlsanity.com/paper/2512.25060/) (2025)
- [Grokking vs. Learning: Same Features, Different Encodings](https://mlsanity.com/paper/2502.01739/) (2025)
- [TRACES: Temporal Recall with Contextual Embeddings for Real-Time Video Anomaly Detection](https://mlsanity.com/paper/2511.00580/) (2025)
- [When Data Falls Short: Grokking Below the Critical Threshold](https://mlsanity.com/paper/2511.04760/) (2025)
- [On the Dynamics of Delayed Generalization in Linear Models](https://mlsanity.com/paper/2602.08302/) (2026)
- [Unveiling Memorization-Generalization Coexistence: A Case Study on Arithmetic Tasks with Label Noise](https://mlsanity.com/paper/2605.18022/) (2026)
- [Batch Normalization Amplifies Memorization and Privacy Risks](https://mlsanity.com/paper/2605.24420/) (2026)
- [From Symmetry to Invariance: Learning Galois Equivalent Representations in Finite Fields](https://mlsanity.com/paper/2608.22513/) (2026)
- [Tracing the Path to Grokking: Embeddings, Dropout, and Network Activation](https://mlsanity.com/paper/2507.11645/) (2025)
- [Learning Pseudorandom Numbers with Transformers: Permuted Congruential Generators, Curricula, and Interpretability](https://mlsanity.com/paper/2510.26792/) (2025)
- [LLMs for Low-Resource Dialect Translation Using Context-Aware Prompting: A Case Study on Sylheti](https://mlsanity.com/paper/2511.21761/) (2025)
- [Latent Algorithmic Structure Precedes Grokking: A Mechanistic Study of ReLU MLPs on Modular Arithmetic](https://mlsanity.com/paper/2603.23784/) (2026)
- [Emergence via Phase Transitions: Mechanism Landscapes and Universal Convergence Across Complex Systems](https://mlsanity.com/paper/2606.07563/) (2026)
- [Shortcut Before Circuit: Document Statistics Time In-Context Conflict Resolution](https://mlsanity.com/paper/2608.24460/) (2026)
- [PrimeScientist: Strategic Allocation of Research Effort in Autonomous Research](https://mlsanity.com/paper/2609.17846/) (2026)

## Abstract (arXiv)

Robust generalization is a major challenge in deep learning, particularly when the number of trainable parameters is very large. In general, it is very difficult to know if the network has memorized a particular set of examples or understood the underlying rule (or both). Motivated by this challenge, we study an interpretable model where generalizing representations are understood analytically, and are easily distinguishable from the memorizing ones. Namely, we consider multi-layer perceptron (MLP) and Transformer architectures trained on modular arithmetic tasks, where (xi · 100%) of labels are corrupted (i.e. some results of the modular operations in the training set are incorrect). We show that (i) it is possible for the network to memorize the corrupted labels and achieve 100% generalization at the same time; (ii) the memorizing neurons can be identified and pruned, lowering the accuracy on corrupted data and improving the accuracy on uncorrupted data; (iii) regularization methods such as weight decay, dropout and BatchNorm force the network to ignore the corrupted data during optimization, and achieve 100% accuracy on the uncorrupted dataset; and (iv) the effect of these regularization methods is (``mechanistically'') interpretable: weight decay and dropout force all the neurons to learn generalizing representations, while BatchNorm de-amplifies the output of memorizing neurons and amplifies the output of the generalizing ones. Finally, we show that in the presence of regularization, the training dynamics involves two consecutive stages: first, the network undergoes grokking dynamics reaching high train and test accuracy; second, it unlearns the memorizing representations, where the train accuracy suddenly jumps from 100% to 100 (1-xi)%.