mlsanityData to 7 Oct 2026

Mechanistic Unlearning: Robust Knowledge Unlearning and Editing via Mechanistic Localization

arXiv 2410.12949 ↗First posted 16 Oct 2024cs.LG cs.CLcited by 34

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Methods for knowledge editing and unlearning in large language models seek to edit or remove undesirable knowledge or capabilities without compromising general language modeling performance. This work investigates how mechanistic interpretability -- which, in part, aims to identify model components (circuits) associated to specific interpretable mechanisms that make up a model capability -- can improve the precision and effectiveness of editing and unlearning. We find a stark difference in unlearning and edit robustness when training components localized by different methods. We highlight an important distinction between methods that localize components based primarily on preserving outputs, and those finding high level mechanisms with predictable intermediate states. In particular, localizing edits/unlearning to components associated with the lookup-table mechanism for factual recall 1) leads to more robust edits/unlearning across different input/output formats, and 2) resists attempts to relearn the unwanted information, while also reducing unintended side effects compared to baselines, on both a sports facts dataset and the CounterFact dataset across multiple models. We also find that certain localized edits disrupt the latent knowledge in the model more than any other baselines, making unlearning more robust to various attacks.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 35 of its 53 references that are in the map. First posted on arXiv 16 Oct 2024.

Builds on

35 of 53 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Do Unlearning Methods Remove Information from Language Model Weights?11 Oct
  2. 2024Intrinsic Test of Unlearning Using Parametric Knowledge Traces17 Jun
  3. 2024Eight Methods to Evaluate Robust Unlearning in LLMs26 Feb
  4. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMs31 Oct
  5. 2023Who's Harry Potter? Approximate Unlearning in LLMs3 Oct

Other references · 30

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  2. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  4. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  5. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  6. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  7. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  8. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  9. 2023Large Language Modelscited by 774
  10. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  11. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  12. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 447
18 more
  1. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  2. 2024tinyBenchmarks: evaluating LLMs with fewer examplescited by 295
  3. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
  4. 2022Surgical Fine-Tuning Improves Adaptation to Distribution Shiftscited by 205
  5. 2023Localizing Model Behavior with Path Patchingcited by 182
  6. 2024A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicitycited by 179
  7. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 156
  8. 2024Tamper-Resistant Safeguards for Open-Weight LLMscited by 132
  9. 2023Task-Specific Skill Localization in Fine-tuned Language Modelscited by 104
  10. 2022Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Modelscited by 56
  11. 2023Eliciting Latent Knowledge from Quirky Language Modelscited by 56
  12. 2023Characterizing Mechanisms for Factual Recall in Language Modelscited by 47
  13. 2024Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMscited by 39
  14. 2023Circuit Breaking: Removing Model Behaviors with Targeted Ablationcited by 27
  15. 2024Large Language Models Relearn Removed Conceptscited by 26
  16. 2023Discovering Knowledge-Critical Subnetworks in Pretrained Language Modelscited by 16
  17. 2024SOPHON: Non-Fine-Tunable Learning to Restrain Task Transferability For Pre-trained Modelscited by 16
  18. 2024In-Context Learning Can Re-learn Forbidden Taskscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 16; 2026: 18.

34 papers.

Sort:

By area:

  1. 2025SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoderscited by 75
  2. 2025Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuningcited by 34
  3. 2025A Comprehensive Survey of Machine Unlearning Techniques for Large Language Modelscited by 30
  4. 2025Editing as Unlearning: Are Knowledge Editing Methods Strong Baselines for Large Language Model Unlearning?cited by 13
  5. 2025CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learnerscited by 9
  6. 2025SoK: Machine Unlearning for Large Language Modelscited by 9
  7. 2025Step-by-Step Reasoning Attack: Revealing 'Erased' Knowledge in Large Language Modelscited by 6
  8. 2026Attention Smoothing Is All You Need For Unlearningcited by 6
  9. 2025Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Modelscited by 5
  10. 2026Is Gradient Ascent Really Necessary? Memorize to Forget for Machine Unlearningcited by 5
  11. 2025DUSK: Do Not Unlearn Shared Knowledgecited by 4
  12. 2025On the creation of narrow AI: hierarchy and nonlocality of neural network skillscited by 4
22 more
  1. 2025AlignSAE: Concept-Aligned Sparse Autoencoderscited by 4
  2. 2026Models Designed to Forget: Machine Unlearning via Key Deletioncited by 3
  3. 2025A Survey on Unlearning in Large Language Modelscited by 2
  4. 2025MIAT: Maneuver-Intention-Aware Transformer for Spatio-Temporal Trajectory Predictioncited by 1
  5. 2025Learning-Time Encoding Shapes Unlearning in LLMscited by 1
  6. 2025LLM Unlearning using Gradient Ratio-Based Influence Estimation and Noise Injectioncited by 1
  7. 2026Detoxifying LLMs via Representation Erasure-Based Preference Optimizationcited by 1
  8. 2026Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attributioncited by 1
  9. 2026Measuring the Depth of LLM Unlearning via Activation Patchingcited by 1
  10. 2026Toward a Benchmark for Controllable Simulation of Imperfect Students with Large Language Modelscited by 1
  11. 2026Exposing the Illusion of Erasure in Knowledge Editing for LLMscited by 1
  12. 2026Measure, Don't Optimize: Forecasting Recovery in LLM Unlearningcited by 1
  13. 2025RippleBench: Capturing Ripple Effects Using Existing Knowledge Repositoriescited by 0
  14. 2026Variance-Reduced (varepsilon,δ)-Unlearning using Forget Set Gradientscited by 0
  15. 2026U-CECE: A Universal Multi-Resolution Framework for Conceptual Counterfactual Explanationscited by 0
  16. 2026MAAT: Multi-phase Adapter-Aware Targeted Unlearningcited by 0
  17. 2026Beyond Static Interpretability: Anticipating Post-SFT Mechanisms from Pre-SFT Parameters for Better Tuningcited by 0
  18. 2026Distance Is Not Enough: Forget-Retain Alignment Gap Predicts LLM Relearning Robustnesscited by 0
  19. 2026The Safeguard Worked. Is the LLM System Safer?cited by 0
  20. 2026Entangled Representations Amplify Collateral Damage in Unlearningcited by 0
  21. 2026Storage Is Not Strategy: State-Conditioned Support Control for LLM Unlearningcited by 0
  22. 2026SIEVE: Selective attention-value Suppression for Vision-Language Models Unlearningcited by 0