# How Learning Governs Unlearning across the Memorization-Generalization Spectrum

arXiv: https://arxiv.org/abs/2610.08577 · HTML: https://mlsanity.com/paper/2610.08577/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 6 of its 38 references that are in the map.
- First posted on arXiv 6 Oct 2026.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 6 of 6)

- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets](https://mlsanity.com/paper/2201.02177/) (2022)
- [Hubble: a Model Suite to Advance the Study of LLM Memorization](https://mlsanity.com/paper/2510.19811/) (2025)
- [CE-U: Cross Entropy Unlearning](https://mlsanity.com/paper/2503.01224/) (2025)
- [Forget What Matters, Keep the Rest: Selective Unlearning of Informative Tokens](https://mlsanity.com/paper/2604.17785/) (2026)
- [Rethinking Federated Unlearning via the Lens of Memorization](https://mlsanity.com/paper/2605.24545/) (2026)

## Abstract (arXiv)

While unlearning seeks to negate undesired capabilities acquired through learning, little research has examined how the way models learn shapes their subsequent unlearning. In this paper, we investigate this connection from the perspectives of memorization and generalization, the two most representative yet competing strategies that models employ during training. We first classify memorization- and generalization-heavy models using grokking in modular addition and compare their responses to unlearning, showing that the latter suffer greater retain damage, i.e., a larger performance drop on the retain set. Furthermore, we conduct a finer-grained analysis by introducing bucketed modular addition, in which the respective contributions of the two strategies can be explicitly controlled across the memorization-generalization spectrum. In this setup, we reaffirm that the same trend persists and is nearly monotonic. We further demonstrate that this relationship also holds in LLM unlearning across verbatim and factual recall settings. Finally, we provide two practical insights for developing better unlearning methods, highlighting the importance of accounting for learning dynamics in unlearning.