mlsanityData to 14 Oct 2026

Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attribution

arXiv 2605.15138 ↗First posted 14 May 2026cs.LG cs.CL cs.ETcited by 1

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Standard unlearning evaluations measure behavioral suppression in full precision, immediately after training, despite every deployed language model being quantized first. Recent work has shown that 4-bit post-training quantization can reverse machine unlearning; we show this is not a tuning artefact but a systematic dual failure: gradient-based methods that achieve meaningful forgetting lose it under compression, while methods that survive quantization barely change the model. Both failures trace to the same root cause: across all baselines, per-parameter updates lie 47-828x below the NF4 quantization bin width; updates diffused across billions of parameters cannot clear quantization bin boundaries, a consequence we formalize as a sparsity-permanence tradeoff. We present MANSU (Mechanistic-Aligned Null-Space Unlearning), which resolves both modes by combining causal circuit attribution to isolate the minimal forget-set subgraph, circuit-restricted null-space projection with a diagonal-Fisher retain bound, and a per-parameter magnitude floor guaranteeing quantization survival by construction. We additionally introduce Circuit Attribution Divergence (CAD), a mechanistic verification metric distinguishing structural erasure from behavioral suppression, a distinction existing metrics cannot make. Across multiple model families and hazard benchmarks, MANSU is the first method to jointly satisfy all four properties with margin on each (meaningful forgetting, retain preservation, non-positive PTQ gap, and structural erasure), while gradient-based baselines recover up to +0.05 accuracy under compression.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 23 of its 26 references that are in the map. First posted on arXiv 14 May 2026.

Builds on

23 of 26 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMs22 May
  2. 2025Does Localization Inform Unlearning? A Rigorous Examination of Local Parameter Attribution for Knowledge Unlearning in Language Models22 May
  3. 2025LLM Unlearning via Neural Activation Redirection11 Feb
  4. 2024Catastrophic Failure of LLM Unlearning via Quantization21 Oct
  5. 2024Mechanistic Unlearning: Robust Knowledge Unlearning and Editing via Mechanistic Localization16 Oct
  6. 2024Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning9 Oct
  7. 2024Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancement29 Sep
  8. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Models8 Jul
  9. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
  10. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  11. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct
  12. 2022Continual Learning and Private Unlearning24 Mar

Other references · 11

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  4. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  5. 2022Mass-Editing Memory in a Transformercited by 1,001
  6. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
  7. 2014New insights and perspectives on the natural gradient methodcited by 443
  8. 2019Limitations of the Empirical Fisher Approximation for Natural Gradient Descentcited by 217
  9. 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 209
  10. 2024Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanismscited by 149
  11. 2025Do All Autoregressive Transformers Remember Facts the Same Way? A Cross-Architecture Analysis of Recall Mechanismscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Quantization-Robust Unlearning through the Lens of Retain-Forget Loss Landscapes Interaction
  2. 2026SAUL: Sharpness-Aware Augmented-Lagrangian Unlearning
  3. 2026Learning What to Forget: Improving LLM Unlearning via Learned Token-Level Importance
  4. 2026Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs
  5. 2026Not Every Change Is Necessary: Recoverable Drift in Large Language Model Unlearning
  6. 2026QUAIL: Quantization Aware Unlearning for Mitigating Misinformation in LLMs
  7. 2026DualOptim+: Bridging Shared and Decoupled Optimizer States for Better Machine Unlearning in Large Language Models
  8. 2026Unmerge: Efficient Machine Unlearning via Task Arithmetic