mlsanityData to 7 Oct 2026

Beyond Forgetting: Representation Misdirection Elicits Controllable Side Behaviors and Capabilities

arXiv 2601.21702 ↗First posted 29 Jan 2026cs.LG cs.CLcited by 1

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

We consider Representation Misdirection (RM), a class of large language model (LLM) unlearning methods that achieve forgetting by redirecting the latent representations of forget-samples toward a target vector. Despite being important, the roles of the target vector used in RM, however, remain underexplored. Here, we approach and revisit RM through the lens of the Linear Representation Hypothesis. Specifically, if one can identify a one-dimensional representation corresponding to a high-level concept, the Linear Representation Hypothesis enables linear operations on this concept vector within the forget-representation space. Under this view, we hypothesize that, beyond forgetting, machine unlearning via RM elicits controllable side effect behaviors and capabilities corresponding to the high-level concept. Our hypothesis is empirically validated across a wide range of concepts and tasks, including controlling unlearned models' truthfulness, sentiment, stereotypical bias, refusal, language, and in-context learning (ICL) tasks. Our findings reveal that this phenomenon could be either a hidden risk if misused or a mechanism that can be harnessed for developing unlearned models that require stronger capabilities and controllable behaviors.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 106 of its 128 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

106 of 128 references.

Closest ancestors · 62

Same direction, newest first

  1. 2025Investigating Model Editing for Unlearning in Large Language Models23 Dec
  2. 2025Feature-Selective Representation Misdirection for Machine Unlearning18 Dec
  3. 2025Hubble: a Model Suite to Advance the Study of LLM Memorization22 Oct
  4. 2025On the Impossibility of Retrain Equivalence in Machine Unlearning18 Oct
  5. 2025Ascent Fails to Forget30 Sep
  6. 2025Dual-Space Smoothness for Robust and Balanced LLM Unlearning27 Sep
  7. 2025Beyond Sharp Minima: Robust LLM Unlearning via Feedback-Guided Multi-Point Optimization24 Sep
  8. 2025Sparse-Autoencoder-Guided Internal Representation Unlearning for Large Language Models19 Sep
  9. 2025The Right to be Forgotten in Pruning: Unveil Machine Unlearning on Sparse Models24 Jul
  10. 2025Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skills15 Jun
  11. 2025SoK: Machine Unlearning for Large Language Models10 Jun
  12. 2025Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuning2 Jun
38 more
  1. 2025Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy31 May
  2. 2025Model Unlearning via Sparse Autoencoder Subspace Guided Projections30 May
  3. 2025Unlearned but Not Forgotten: Data Extraction after Exact Unlearning in LLM30 May
  4. 2025BLUR: A Benchmark for LLM Unlearning Robust to Forget-Retain Overlap28 May
  5. 2025Editing as Unlearning: Are Knowledge Editing Methods Strong Baselines for Large Language Model Unlearning?26 May
  6. 2025Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMs22 May
  7. 2025SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs11 Apr
  8. 2025Exact Unlearning of Finetuning Data via Model Merging at Scale6 Apr
  9. 2025Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyond26 Feb
  10. 2025A General Framework to Enhance Fine-tuning-based LLM Unlearning25 Feb
  11. 2025LLM Unlearning via Neural Activation Redirection11 Feb
  12. 2025Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond7 Feb
  13. 2025Agents Are All You Need for LLM Unlearning1 Feb
  14. 2025Improving LLM Unlearning Robustness via Random Perturbations31 Jan
  15. 2025Open Problems in Machine Unlearning for AI Safety9 Jan
  16. 2024Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy and Research9 Dec
  17. 2024Unified Parameter-Efficient Unlearning for LLMs30 Nov
  18. 2024Delta-Influence: Unlearning Poisons via Influence Functions20 Nov
  19. 2024Does Unlearning Truly Unlearn? A Black Box Evaluation of LLM Unlearning Methods18 Nov
  20. 2024Applying sparse autoencoders to unlearn knowledge in language models25 Oct
  21. 2024When Machine Unlearning Meets Retrieval-Augmented Generation (RAG): Keep Secret or Forget Knowledge?20 Oct
  22. 2024Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy12 Oct
  23. 2024Do Unlearning Methods Remove Information from Language Model Weights?11 Oct
  24. 2024A Closer Look at Machine Unlearning for Large Language Models10 Oct
  25. 2024Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning9 Oct
  26. 2024Position: LLM Unlearning Benchmarks are Weak Measures of Progress3 Oct
  27. 2024An Adversarial Perspective on Machine Unlearning for AI Safety26 Sep
  28. 2024On Effects of Steering Latent Representation for Large Language Model Unlearning12 Aug
  29. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Models8 Jul
  30. 2024Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning19 Jun
  31. 2024Large Language Model Unlearning via Embedding-Corrupted Prompts12 Jun
  32. 2024Large Scale Knowledge Washing26 May
  33. 2024SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning28 Apr
  34. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
  35. 2024Second-Order Information Matters: Revisiting Machine Unlearning for Large Language Models13 Mar
  36. 2024Guardrail Baselines for Unlearning in LLMs5 Mar
  37. 2024Dissecting Language Models: Machine Unlearning via Selective Pruning2 Mar
  38. 2024Rethinking Machine Unlearning for Large Language Models13 Feb

50 newest of 62 All 62 (JSON)

Other references · 44

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  6. 2024Qwen2.5 Technical Reportcited by 4,713
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  8. 2016Pointer Sentinel Mixture Modelscited by 2,992
  9. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  10. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  11. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  12. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
32 more
  1. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  2. 2014How transferable are features in deep neural networks?cited by 1,216
  3. 2019Machine Unlearningcited by 1,020
  4. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  5. 2022Toy Models of Superpositioncited by 928
  6. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  7. 2023Steering Language Models With Activation Engineeringcited by 824
  8. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  9. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  10. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  11. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  12. 2023Zephyr: Direct Distillation of LM Alignmentcited by 498
  13. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  14. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  15. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  16. 2023Studying Large Language Model Generalization with Influence Functionscited by 311
  17. 2023Language Models Represent Space and Timecited by 306
  18. 2023In-Context Learning Creates Task Vectorscited by 302
  19. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
  20. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  21. 2023Linear Representations of Sentiment in Large Language Modelscited by 157
  22. 2023Composing Parameter-Efficient Modules with Arithmetic Operationscited by 155
  23. 2024GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solverscited by 155
  24. 2024The Geometry of Categorical and Hierarchical Concepts in Large Language Modelscited by 151
  25. 2024Tamper-Resistant Safeguards for Open-Weight LLMscited by 132
  26. 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 109
  27. 2024Representation Noising: A Defence Mechanism Against Harmful Finetuningcited by 93
  28. 2023Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinkingcited by 78
  29. 2023DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Modelscited by 75
  30. 2023Concept Algebra for (Score-Based) Text-Controlled Generative Modelscited by 59
  31. 2024Large Language Models Relearn Removed Conceptscited by 26
  32. 2024Tradeoffs Between Alignment and Helpfulness in Language Models with Steering Methodscited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Unlearning Deceptive Behaviors in LLMs with Contrastive Forget Setscited by 0