# LLM Unlearning via Neural Activation Redirection

arXiv: https://arxiv.org/abs/2502.07218 · HTML: https://mlsanity.com/paper/2502.07218/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 59 of its 79 references that are in the map.
- First posted on arXiv 11 Feb 2025.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 30 of 59)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023)
- [Editing Models with Task Arithmetic](https://mlsanity.com/paper/2212.04089/) (2022)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models](https://mlsanity.com/paper/2310.04451/) (2023)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [A Generalization of Transformer Networks to Graphs](https://mlsanity.com/paper/2012.09699/) (2020)
- [CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data](https://mlsanity.com/paper/1911.00359/) (2019)
- [TOFU: A Task of Fictitious Unlearning for LLMs](https://mlsanity.com/paper/2401.06121/) (2024)
- [The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning](https://mlsanity.com/paper/2403.03218/) (2024)
- [Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning](https://mlsanity.com/paper/2404.05868/) (2024)
- [SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks](https://mlsanity.com/paper/2310.03684/) (2023)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation](https://mlsanity.com/paper/2310.06987/) (2023)
- [Reducing Transformer Depth on Demand with Structured Dropout](https://mlsanity.com/paper/1909.11556/) (2019)
- [Knowledge Unlearning for Mitigating Privacy Risks in Language Models](https://mlsanity.com/paper/2210.01504/) (2022)
- [Scalable Extraction of Training Data from (Production) Language Models](https://mlsanity.com/paper/2311.17035/) (2023)
- [Remember What You Want to Forget: Algorithms for Machine Unlearning](https://mlsanity.com/paper/2103.03279/) (2021)
- [Studying Large Language Model Generalization with Influence Functions](https://mlsanity.com/paper/2308.03296/) (2023)
- [A Survey of Machine Unlearning](https://mlsanity.com/paper/2209.02299/) (2022)
- [LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding](https://mlsanity.com/paper/2404.16710/) (2024)

## Built on by (most cited first, 30 of 41)

- [LLM Unlearning with LLM Beliefs](https://mlsanity.com/paper/2510.19422/) (2025)
- [Editing as Unlearning: Are Knowledge Editing Methods Strong Baselines for Large Language Model Unlearning?](https://mlsanity.com/paper/2505.19855/) (2025)
- [SoK: Machine Unlearning for Large Language Models](https://mlsanity.com/paper/2506.09227/) (2025)
- [AbbIE: Autoregressive Block-Based Iterative Encoder for Efficient Sequence Modeling](https://mlsanity.com/paper/2507.08567/) (2025)
- [DRAGON: Guard LLM Unlearning in Context via Negative Detection and Reasoning](https://mlsanity.com/paper/2511.05784/) (2025)
- [Hallucination reduction with CASAL: Contrastive Activation Steering For Amortized Learning](https://mlsanity.com/paper/2510.02324/) (2025)
- [UniErase: Towards Balanced and Precise Unlearning in Language Models](https://mlsanity.com/paper/2505.15674/) (2025)
- [Generative Model Unlearning: A Survey through Target Events, Unlearning Operators, and Evaluation Protocols](https://mlsanity.com/paper/2507.19894/) (2025)
- [A Survey on Unlearning in Large Language Models](https://mlsanity.com/paper/2510.25117/) (2025)
- [iSeal: Encrypted Fingerprinting for Reliable LLM Ownership Verification](https://mlsanity.com/paper/2511.08905/) (2025)
- [LLM Unlearning Should Be Form-Independent](https://mlsanity.com/paper/2506.07795/) (2025)
- [SEAT: Sparse Entity-Aware Tuning for Knowledge Adaptation while Preserving Epistemic Abstention](https://mlsanity.com/paper/2506.14387/) (2025)
- [Position: The Term "Machine Unlearning" Is Overused in LLMs](https://mlsanity.com/paper/2606.27379/) (2026)
- [Per-parameter Task Arithmetic for Unlearning in Large Language Models](https://mlsanity.com/paper/2601.22030/) (2026)
- [Mitigating Memorization Where It Happens](https://mlsanity.com/paper/2602.08901/) (2026)
- [KUDA: Knowledge Unlearning by Deviating Representation for Large Language Models](https://mlsanity.com/paper/2602.19275/) (2026)
- [ALTER: Asymmetric LoRA for Token-Entropy-Guided Unlearning of LLMs](https://mlsanity.com/paper/2603.01792/) (2026)
- [Distinguishable Deletion: Unifying Knowledge Erasure and Refusal for Large Language Model Unlearning](https://mlsanity.com/paper/2605.16776/) (2026)
- [Divergence Decoding: Inference-Time Unlearning via Auxiliary Models](https://mlsanity.com/paper/2605.31293/) (2026)
- [Multilingual Unlearning in LLMs: Transfer, Dynamics, and Reversibility](https://mlsanity.com/paper/2606.03291/) (2026)
- [Beyond Forgetting: Representation Misdirection Elicits Controllable Side Behaviors and Capabilities](https://mlsanity.com/paper/2601.21702/) (2026)
- [Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attribution](https://mlsanity.com/paper/2605.15138/) (2026)
- [How LLM Task-Adaptation Reshapes Alignment: A Multi-dimensional Study of Behavioral and Representational Drift](https://mlsanity.com/paper/2607.22676/) (2026)
- [Sparse Readout Prism: Explaining Logit-Lens Scores in Features Instead of Tokens](https://mlsanity.com/paper/2609.01936/) (2026)
- [Representation-Aware Unlearning via Activation Signatures: From Suppression to Entity-Signature Erasure](https://mlsanity.com/paper/2601.10566/) (2026)
- [How to Take a Memorable Picture? Empowering Users with Actionable Feedback](https://mlsanity.com/paper/2602.21877/) (2026)
- [MPU: Towards Secure and Privacy-Preserving Knowledge Unlearning for Large Language Models](https://mlsanity.com/paper/2602.23798/) (2026)
- [Fuel Gauge: Estimating Chain-of-Thought Length Ahead of Time in Large Multimodal Models](https://mlsanity.com/paper/2603.10335/) (2026)
- [Referring Multiple Regions with Large Multimodal Models via Contextual Latent Steering](https://mlsanity.com/paper/2605.01827/) (2026)
- [BARRIER: Bounded Activation Regions for Robust Information Erasure](https://mlsanity.com/paper/2605.15737/) (2026)

## Abstract (arXiv)

The ability to selectively remove knowledge from LLMs is highly desirable. However, existing methods often struggle with balancing unlearning efficacy and retain model utility, and lack controllability at inference time to emulate base model behavior as if it had never seen the unlearned data. In this paper, we propose LUNAR, a novel unlearning method grounded in the Linear Representation Hypothesis and operates by redirecting the representations of unlearned data to activation regions that expresses its inability to answer. We show that contrastive features are not a prerequisite for effective activation redirection, and LUNAR achieves state-of-the-art unlearning performance and superior controllability. Specifically, LUNAR achieves between 2.9x and 11.7x improvement in the combined unlearning efficacy and model utility score (Deviation Score) across various base models and generates coherent, contextually appropriate responses post-unlearning. Moreover, LUNAR effectively reduces parameter updates to a single down-projection matrix, a novel design that significantly enhances efficiency by 20x and robustness. Finally, we demonstrate that LUNAR is robust to white-box adversarial attacks and versatile in real-world scenarios, including handling sequential unlearning requests.