# Forget Narrowly, Retain Broadly: Unlearning as an Asymmetric Generalization Problem

arXiv: https://arxiv.org/abs/2607.09236 · HTML: https://mlsanity.com/paper/2607.09236/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 8 of its 52 references that are in the map.
- First posted on arXiv 10 Jul 2026.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 8 of 8)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Who's Harry Potter? Approximate Unlearning in LLMs](https://mlsanity.com/paper/2310.02238/) (2023)
- [Ministral 3](https://mlsanity.com/paper/2601.08584/) (2026)
- [Knowledge Sanitization of Large Language Models](https://mlsanity.com/paper/2309.11852/) (2023)
- [LUME: LLM Unlearning with Multitask Evaluations](https://mlsanity.com/paper/2502.15097/) (2025)
- [BLUR: A Benchmark for LLM Unlearning Robust to Forget-Retain Overlap](https://mlsanity.com/paper/2506.15699/) (2025)
- [Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMs](https://mlsanity.com/paper/2509.02820/) (2025)
- [RippleBench: Capturing Ripple Effects Using Existing Knowledge Repositories](https://mlsanity.com/paper/2512.04144/) (2025)

## Abstract (arXiv)

Machine unlearning in LLMs is the targeted removal of specific knowledge while preserving all other capabilities, critical for privacy and safety. Yet existing benchmarks measure it unreliably. They miss knowledge that resurfaces under paraphrased or indirect queries, a failure we call under-forgetting, and lack the semantic, syntactic, and lexical probes needed to verify that unrelated knowledge is preserved, a failure we call over-forgetting. Both failures reflect an asymmetric generalization problem. Forget evaluation must cover diverse query formulations of the same target facts, testing whether forgetting holds beyond exact training prompts. Retain evaluation must probe a far larger and implicitly defined set, namely every fact disjoint from the forget target. The retain set thus defines the effective forget set, yet current datasets provide no fine-grained annotation of this forget-retain boundary. We address this with SUITE, an evaluation protocol and training corpus that captures forget-retain structure for real-world factual domains. Methods trained on SUITE improve substantially, showing that training data is as important as algorithmic design. Building on the obtained insights, we introduce JensUn++, an unlearning algorithm that achieves the best forget-retain utility trade-off across three LLMs, in both sequential and joint unlearning settings. Code and datasets are available at https://amitpeleg.github.io/forget-narrowly-retain-broadly