mlsanityData to 14 Oct 2026

RA2: Retain-Anchored Attraction Preserves Forget-Adjacent Utility in LLM Unlearning

arXiv 2610.19385 ↗First posted 14 Oct 2026cs.LGcited by 0

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Large Language Model (LLM) unlearning is often evaluated primarily by target suppression and clean retain utility. These metrics can miss a local failure mode: a model may perform well on standard retain prompts, yet degrade on benign reasoning when prompts lie near the forgotten domain. This phenomenon is termed forget-adjacent utility loss. A fundamental question is: where should forget representations go after they leave the removed behavior? Successful unlearning depends not only on whether forget-conditioned representations depart from the harmful computation, but also on the direction in which they are redirected afterward. We propose RA2 (Retain-Anchored Attraction), a latent-space unlearning method that redirects forget-token states toward nearby retain-supported representations while preserving higher-layer behavior on retain data. On WMDP and MUSE, RA2 yields higher forget-adjacent utility than strong unlearning baselines at similar broad-utility levels. These results show that the migration direction of forget representations materially affects benign behavior near the forget boundary. Our code is available at https://github.com/ZacLing/retainanchoredattraction.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 55 of its 67 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

55 of 67 references.

Closest ancestors · 39

Same direction, newest first

  1. 2026Machine Unlearning under Retain-Forget Entanglement27 Mar
  2. 2026KUDA: Knowledge Unlearning by Deviating Representation for Large Language Models22 Feb
  3. 2026Erase at the Core: Representation Unlearning for Machine Unlearning5 Feb
  4. 2026Contrastive Representation Shaping for LLM Unlearning29 Jan
  5. 2025Feature-Selective Representation Misdirection for Machine Unlearning18 Dec
  6. 2025A Survey on Unlearning in Large Language Models29 Oct
  7. 2025Forget to Know, Remember to Use: Context-Aware Unlearning for Large Language Models20 Oct
  8. 2025LLM Unlearning Under the Microscope: A Full-Stack View on Methods and Metrics8 Oct
  9. 2025Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMs2 Sep
  10. 2025The Right to be Forgotten in Pruning: Unveil Machine Unlearning on Sparse Models24 Jul
  11. 2025Unlearning Isn't Invisible: Detecting Unlearning Traces in LLMs from Model Outputs16 Jun
  12. 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics14 Jun
27 more
  1. 2025BLUR: A Benchmark for LLM Unlearning Robust to Forget-Retain Overlap28 May
  2. 2025Harry Potter is Still Here! Probing Knowledge Leakage in Targeted Unlearned Large Language Models via Automated Adversarial Prompting22 May
  3. 2025OBLIVIATE: Robust and Practical Machine Unlearning for Large Language Models7 May
  4. 2025SUV: Scalable Large Language Model Copyright Compliance with Regularized Selective Unlearning29 Mar
  5. 2025A Comprehensive Survey of Machine Unlearning Techniques for Large Language Models22 Feb
  6. 2025Which Retain Set Matters for LLM Unlearning? A Case Study on Entity Unlearning17 Feb
  7. 2025Improving LLM Unlearning Robustness via Random Perturbations31 Jan
  8. 2024Towards Robust Evaluation of Unlearning in LLMs via Data Transformations23 Nov
  9. 2024Catastrophic Failure of LLM Unlearning via Quantization21 Oct
  10. 2024LLM Unlearning via Loss Adjustment with Only Forget Data14 Oct
  11. 2024A Closer Look at Machine Unlearning for Large Language Models10 Oct
  12. 2024Position: LLM Unlearning Benchmarks are Weak Measures of Progress3 Oct
  13. 2024An Adversarial Perspective on Machine Unlearning for AI Safety26 Sep
  14. 2024UnUnlearning: Unlearning is not sufficient for content regulation in advanced generative AI27 Jun
  15. 2024Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning19 Jun
  16. 2024Soft Prompting for Unlearning in Large Language Models17 Jun
  17. 2024Intrinsic Test of Unlearning Using Parametric Knowledge Traces17 Jun
  18. 2024Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference12 Jun
  19. 2024Large Language Model Unlearning via Embedding-Corrupted Prompts12 Jun
  20. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
  21. 2024Digital Forgetting in Large Language Models: A Survey of Unlearning Methods2 Apr
  22. 2024Eight Methods to Evaluate Robust Unlearning in LLMs26 Feb
  23. 2024Towards Safer Large Language Models through Machine Unlearning15 Feb
  24. 2024Rethinking Machine Unlearning for Large Language Models13 Feb
  25. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  26. 2023Large Language Model Unlearning14 Oct
  27. 2023Who's Harry Potter? Approximate Unlearning in LLMs3 Oct

Other references · 16

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  3. 2016Pointer Sentinel Mixture Modelscited by 3,096
  4. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  6. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  7. 2022Editing Models with Task Arithmeticcited by 1,120
  8. 2019Machine Unlearningcited by 1,061
  9. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
  10. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 173
  11. 2025Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMscited by 42
  12. 2024Accurate and Nuanced Open-QA Evaluation Through Textual Entailmentcited by 13
4 more
  1. 2024The Mosaic Memory of Large Language Modelscited by 10
  2. 2025From Narrow Unlearning to Emergent Misalignment: Causes, Consequences, and Containment in LLMscited by 6
  3. 2026Your Language Model Secretly Contains Personality Subnetworkscited by 5
  4. 2025Representation Interventions Enable Lifelong Knowledge Memory Control in LLMscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning What to Forget: Improving LLM Unlearning via Learned Token-Level Importance
  2. 2025LLM Unlearning on Noisy Forget Sets: A Study of Incomplete, Rewritten, and Watermarked Data
  3. 2025Understanding the Dilemma of Unlearning for Large Language Models
  4. 2026Do LLMs Really Forget? Hidden-State Leakage in Model Unlearning and How to Fix it
  5. 2026Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs
  6. 2026Not Every Change Is Necessary: Recoverable Drift in Large Language Model Unlearning
  7. 2026Crossing the Margin Cliff: Toward Relearn-Robust LLM Unlearning via Margin Calibration
  8. 2025RULE: Reinforcement UnLEarning Achieves Forget-Retain Pareto Optimality