mlsanityData to 14 Oct 2026

Causal Routing for Unlearning

arXiv 2609.34475 ↗First posted 28 Sep 2026cs.LG cs.AIcited by 0

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

LLMs cannot forget the way we delete a file. Strangely, we are asked to remove something that was never put anywhere in particular. What the model took from a piece of text is now smeared across billions of weights. Existing methods rewrite all of them to change one thing, and none of them say which part produced that change. To address this, we introduce Causal Routing for Unlearning (CRU) by asking where the concept is expressed in the model and suppressing only that part. One untrained forward pass over the forget set ranks neurons by how their activations vary. Then, small routing modules on those neurons gate and suppress only the concepts that need to be forgotten. In CRU, the base model is frozen, and any change in behavior is caused only by the gated neurons; hence, why the routing is causal. Due to our parameter efficiency (only ~0.01% as many parameters as the base model), unlearning a concept costs 14 GiB, whereas the baselines require 71 GiB. On TOFU, CRU is indistinguishable from the retained model (p > 0.05, KS test) and is never Pareto-dominated, whereas every compared baseline matches its forgetting on the larger-forget batches only by collapsing utility. On RWKU, it achieves an adversarial-probe recall of 0.052, compared to 0.250 for the strongest baseline, meaning the knowledge is gone, not merely harder to reach. Thus, deciding on the intervention at query time, rather than fixing it beforehand, is the axis along which we argue that unlearning should proceed.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 36 of its 46 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

36 of 46 references.

Closest ancestors · 17

Same direction, newest first

  1. 2026Inference-Time Machine Unlearning via Gated Activation Redirection12 May
  2. 2026Position: The Term "Machine Unlearning" Is Overused in LLMs8 May
  3. 2026Reinforcement Unlearning via Group Relative Policy Optimization28 Jan
  4. 2025CLUE: Conflict-guided Localization for LLM Unlearning Framework25 Sep
  5. 2025SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs11 Apr
  6. 2025LLM Unlearning via Neural Activation Redirection11 Feb
  7. 2024Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning9 Oct
  8. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Models8 Jul
  9. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
  10. 2024Rethinking Machine Unlearning for Large Language Models13 Feb
  11. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  12. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation19 Oct
5 more
  1. 2023Large Language Model Unlearning14 Oct
  2. 2023In-Context Unlearning: Language Models as Few Shot Unlearners11 Oct
  3. 2022Continual Learning and Private Unlearning24 Mar
  4. 2021On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning22 Oct
  5. 2021Remember What You Want to Forget: Algorithms for Machine Unlearning4 Mar

Other references · 19

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  3. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  5. 2020GLU Variants Improve Transformercited by 1,804
  6. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  7. 2019Machine Unlearningcited by 1,061
  8. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  9. 2023Steering Language Models With Activation Engineeringcited by 922
  10. 2016Language Modeling with Gated Convolutional Networkscited by 702
  11. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  12. 2019Certified Data Removal from Machine Learning Modelscited by 535
7 more
  1. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 376
  2. 2024Programming Refusal with Conditional Activation Steeringcited by 206
  3. 2021All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Qualitycited by 156
  4. 2024Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Modelscited by 126
  5. 2022Finding Skill Neurons in Pre-trained Transformer-based Language Modelscited by 66
  6. 2023Neuron-Level Knowledge Attribution in Large Language Modelscited by 37
  7. 2024Activation Scaling for Steering and Interpreting Language Modelscited by 35

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024CaMU: Disentangling Causal Effects in Deep Model Unlearning
  2. 2025SIMU: Selective Influence Machine Unlearning
  3. 2026Trajectory-Guided Forget-Recover Network for Continual LLM Unlearning
  4. 2025RULE: Reinforcement UnLEarning Achieves Forget-Retain Pareto Optimality
  5. 2025On the Impossibility of Retrain Equivalence in Machine Unlearning
  6. 2026RA2: Retain-Anchored Attraction Preserves Forget-Adjacent Utility in LLM Unlearning
  7. 2026Unmerge: Efficient Machine Unlearning via Task Arithmetic
  8. 2025Understanding Machine Unlearning Through the Lens of Mode Connectivity