mlsanityData to 14 Oct 2026

Applying sparse autoencoders to unlearn knowledge in language models

arXiv 2410.19278 ↗First posted 25 Oct 2024cs.LG cs.AIcited by 59

On the map: LLM safety and security → Machine unlearning

also in 1 more
Abstract (arXiv)

We investigate whether sparse autoencoders (SAEs) can be used to remove knowledge from language models. We use the biology subset of the Weapons of Mass Destruction Proxy dataset and test on the gemma-2b-it and gemma-2-2b-it language models. We demonstrate that individual interpretable biology-related SAE features can be used to unlearn a subset of WMDP-Bio questions with minimal side-effects in domains other than biology. Our results suggest that negative scaling of feature activations is necessary and that zero ablating features is ineffective. We find that intervening using multiple SAE features simultaneously can unlearn multiple different topics, but with similar or larger unwanted side-effects than the existing Representation Misdirection for Unlearning technique. Current SAE quality or intervention techniques would need to improve to make SAE-based unlearning comparable to the existing fine-tuning based techniques.

Key facts. Cited by 59 papers in the map (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 10 of its 17 references that are in the map. First posted on arXiv 25 Oct 2024.

Builds on

10 of 17 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024An Adversarial Perspective on Machine Unlearning for AI Safety26 Sep
  2. 2023Who's Harry Potter? Approximate Unlearning in LLMs3 Oct

Other references · 8

  1. 2016Pointer Sentinel Mixture Modelscited by 3,096
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  3. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  4. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  5. 2024Scaling and evaluating sparse autoencoderscited by 553
  6. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  7. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  8. 2024Improving Alignment and Robustness with Circuit Breakerscited by 322

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 38; 2026: 19.

59 papers.

Sort:

By area:

  1. 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilitycited by 124
  2. 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probingcited by 115
  3. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 95
  4. 2025SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoderscited by 80
  5. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelscited by 76
  6. 2025Enhancing Automated Interpretability with Output-Centric Feature Descriptionscited by 41
  7. 2025Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuningcited by 39
  8. 2025Understanding the Repeat Curse in Large Language Models from a Feature Perspectivecited by 37
  9. 2025Steering off Course: Reliability Challenges in Steering Language Modelscited by 34
  10. 2025SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMscited by 22
  11. 2026Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Modelscited by 21
  12. 2025Position: Use Sparse Autoencoders to Discover Unknownscited by 19
38 more
  1. 2025Model Unlearning via Sparse Autoencoder Subspace Guided Projectionscited by 18
  2. 2024Tracking the Feature Dynamics in LLM Training: A Mechanistic Studycited by 16
  3. 2025SAUCE: Selective Concept Unlearning in Vision-Language Models with Sparse Autoencoderscited by 13
  4. 2025Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Modelscited by 13
  5. 2025Precise In-Parameter Concept Erasure in Large Language Modelscited by 11
  6. 2025SoK: Machine Unlearning for Large Language Modelscited by 11
  7. 2025CRISP: Persistent Concept Unlearning via Sparse Autoencoderscited by 11
  8. 2025Jacobian Sparse Autoencoders: Sparsify Computations, Not Just Activationscited by 10
  9. 2025AbsTopK: Rethinking Sparse Autoencoders For Bidirectional Featurescited by 9
  10. 2025Hypothesis-Driven Feature Manifold Analysis in LLMs via Supervised Multi-Dimensional Scalingcited by 8
  11. 2026AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoderscited by 8
  12. 2025EasyEdit2: An Easy-to-use Steering Framework for Editing Large Language Modelscited by 7
  13. 2026DLM-Scope: Mechanistic Interpretability of Diffusion Language Models via Sparse Autoencoderscited by 7
  14. 2026Understanding Emergent Misalignment via Feature Superposition Geometrycited by 7
  15. 2025Don't Forget It! Conditional Sparse Autoencoder Clamping Works for Unlearningcited by 6
  16. 2025Towards Interpretability Without Sacrifice: Faithful Dense Layer Decomposition with Mixture of Decoderscited by 5
  17. 2025Signal in the Noise: Polysemantic Interference Transfers and Predicts Cross-Model Influencecited by 4
  18. 2025Generative Model Unlearning: A Survey through Target Events, Unlearning Operators, and Evaluation Protocolscited by 4
  19. 2025Measuring Sparse Autoencoder Feature Sensitivitycited by 4
  20. 2025A Comparative Analysis of Sparse Autoencoder and Activation Difference in Language Model Steeringcited by 4
  21. 2025A Survey on Unlearning in Large Language Modelscited by 4
  22. 2025AlignSAE: Concept-Aligned Sparse Autoencoderscited by 4
  23. 2025Ensembling Sparse Autoencoderscited by 3
  24. 2025Resurrecting the Salmon: Rethinking Mechanistic Interpretability with Domain-Specific Sparse Autoencoderscited by 3
  25. 2026Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselinescited by 3
  26. 2025Are Sparse Autoencoders Useful for Java Function Bug Detection?cited by 2
  27. 2025Can SAEs reveal and mitigate racial biases of LLMs in healthcare?cited by 2
  28. 2026Finding the Translation Switch: Discovering and Exploiting the Task-Initiation Features in LLMscited by 2
  29. 2026Sparse CLIP: Co-Optimizing Interpretability and Performance in Contrastive Learningcited by 2
  30. 2026Tracing the Thought of a Grandmaster-level Chess-Playing Transformercited by 2
  31. 2025Concept-Guided Interpretability via Neural Chunkingcited by 1
  32. 2025Automated Neuron Labelling Enables Generative Steering and Interpretability in Protein Language Modelscited by 1
  33. 2025Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generationcited by 1
  34. 2025Rethinking Sparse Autoencoders: Select-and-Project for Fairness and Control from Encoder Features Alonecited by 1
  35. 2025ConceptViz: A Visual Analytics Approach for Exploring Concepts in Large Language Modelscited by 1
  36. 2026Beyond Forgetting: Representation Misdirection Elicits Controllable Side Behaviors and Capabilitiescited by 1
  37. 2026TEVI: Text-Conditioned Editing of Visual Representations via Sparse Autoencoders for Improved Vision-Language Alignmentcited by 1
  38. 2026SAE Interventions are Unreliable: Post-Intervention Recovery of Suppressed Behaviorcited by 1

50 most cited of 59 All 59 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Sparse-Autoencoder-Guided Internal Representation Unlearning for Large Language Models
  2. 2025Do Sparse Autoencoders Generalize? A Case Study of Answerability
  3. 2025Sparse Autoencoder Features for Classifications and Transferability
  4. 2024CodeUnlearn: Amortized Zero-Shot Machine Unlearning in Language Models Using Discrete Concept
  5. 2025Sparse Autoencoders Trained on the Same Data Learn Different Features
  6. 2025Beyond Redundancy: Diverse and Specialized Multi-Expert Sparse Autoencoder
  7. 2024Automatically Interpreting Millions of Features in Large Language Models
  8. 2025Sparse Autoencoders Can Capture Language-Specific Concepts Across Diverse Languages