PreUnlearn: Auditing Collateral Knowledge Damage Before Large Language Model Unlearning
On the map: LLM safety and security → Machine unlearning
Abstract (arXiv)
Machine unlearning for large language models (LLMs) aims to remove specified knowledge while preserving the rest of the model's capabilities. However, the boundary between knowledge to forget and knowledge to retain is often unclear, since related and even distant information may be entangled in the model. In this paper, we study LLM unlearning from a data-centric perspective and measure how unlearning effects propagate from the forget set to same-domain and distant-domain knowledge not after but before unlearning. We find a consistent decay pattern: collateral damage is strongest near the forget set, weakens with semantic distance, but does not disappear at domain boundaries. We further ask whether such damage can be audited before unlearning is executed. We formulate forget-set auditing as a pre-unlearning prediction task and analyze which data features are most predictive of downstream damage. Our results show that interaction features between the forget set and evaluation set provide the strongest signals, suggesting that collateral damage is partly reflected in data geometry before model updates occur. These findings position forget-set auditing as an early warning tool for identifying risky unlearning runs and designing more reliable unlearning procedures. Code and data are available at https://github.com/BartSu/PreUnlearn.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 16 of its 22 references that are in the map. First posted on arXiv 16 Jun 2026.
Builds on
16 of 22 references.
Closest ancestors · 8
Same direction, newest first
- 2025Probing Knowledge Holes in Unlearned LLMs27 Oct
- 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics14 Jun
- 2024Benchmarking Vision Language Model Unlearning via Fictitious Facial Identity Dataset5 Nov
- 2024UNDIAL: Self-Distillation with Adjusted Logits for Robust Unlearning in Large Language Models15 Feb
- 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
- 2023Who's Harry Potter? Approximate Unlearning in LLMs3 Oct
- 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct
- 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearning27 Sep
Other references · 8
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2016Pointer Sentinel Mixture Modelscited by 3,096
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
- 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
- 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
- 2020MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformerscited by 245
- 2026How Order-Sensitive Are LLMs? OrderProbe for Deterministic Structural Reconstructioncited by 9
- 2024A Curious Case of Searching for the Correlation between Training Data and Adversarial Robustness of Transformer Textual Modelscited by 5
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Understanding the Dilemma of Unlearning for Large Language Models
- 2026Measure, Don't Optimize: Forecasting Recovery in LLM Unlearning
- 2025LLM Unlearning on Noisy Forget Sets: A Study of Incomplete, Rewritten, and Watermarked Data
- 2026Confess What You Know: Forget-Set Misalignment with Model Knowledge in LLM Unlearning
- 2025Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMs
- 2025BLUR: A Benchmark for LLM Unlearning Robust to Forget-Retain Overlap
- 2025Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMs
- 2025The Erasure Illusion: Stress-Testing the Generalization of LLM Forgetting Evaluation