mlsanityData to 14 Oct 2026

Are Sparse Autoencoders Useful? A Case Study in Sparse Probing

arXiv 2502.16681 ↗First posted 23 Feb 2025cs.LG cs.AIcited by 115

On the map: Mechanistic interpretability → SAE features and their applications

Abstract (arXiv)

Sparse autoencoders (SAEs) are a popular method for interpreting concepts represented in large language model (LLM) activations. However, there is a lack of evidence regarding the validity of their interpretations due to the lack of a ground truth for the concepts used by an LLM, and a growing number of works have presented problems with current SAEs. One alternative source of evidence would be demonstrating that SAEs improve performance on downstream tasks beyond existing baselines. We test this by applying SAEs to the real-world task of LLM activation probing in four regimes: data scarcity, class imbalance, label noise, and covariate shift. Due to the difficulty of detecting concepts in these challenging settings, we hypothesize that SAEs' basis of interpretable, concept-level latents should provide a useful inductive bias. However, although SAEs occasionally perform better than baselines on individual datasets, we are unable to design ensemble methods combining SAEs with baselines that consistently outperform ensemble methods solely using baselines. Additionally, although SAEs initially appear promising for identifying spurious correlations, detecting poor dataset quality, and training multi-token probes, we are able to achieve similar results with simple non-SAE baselines as well. Though we cannot discount SAEs' utility on other tasks, our findings highlight the shortcomings of current SAEs and the need to rigorously evaluate interpretability methods on downstream tasks with strong baselines.

Key facts. Cited by 115 papers in the map (data through 14 Oct 2026). Placed in SAE features and their applications, one of 13 directions in Mechanistic interpretability. Builds on 42 of its 69 references that are in the map. First posted on arXiv 23 Feb 2025.

Builds on

42 of 69 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders28 Jan
  2. 2024Improving Steering Vectors by Targeting Sparse Autoencoder Features4 Nov
  3. 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders27 Oct
  4. 2024Applying sparse autoencoders to unlearn knowledge in language models25 Oct
  5. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders22 Sep
  6. 2024Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Models31 Jul
  7. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders19 Jul
  8. 2024Improving Dictionary Learning with Gated Sparse Autoencoders24 Apr
  9. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Models15 Sep

Other references · 33

  1. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  2. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  3. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  4. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  5. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  6. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  7. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  8. 2018Sanity Checks for Saliency Mapscited by 1,156
  9. 2018Neural Network Acceptability Judgmentscited by 958
  10. 2017Crowdsourcing Multiple Choice Science Questionscited by 765
  11. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 736
  12. 2020Aligning AI With Shared Human Valuescited by 723
21 more
  1. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  2. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  3. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  4. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 334
  5. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
  6. 2023Language Models Represent Space and Timecited by 316
  7. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 212
  8. 2016Linear Algebraic Structure of Word Senses, with Applications to Polysemycited by 165
  9. 2019"Going on a vacation" takes longer than "Going for a walk": A Study of Temporal Commonsense Understandingcited by 123
  10. 2022CommonsenseQA 2.0: Exposing the Limits of AI through Gamificationcited by 109
  11. 2021Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factorscited by 108
  12. 2021An Interpretability Illusion for BERTcited by 97
  13. 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 79
  14. 2024Identifying Functionally Important Features with End-to-End Sparse Dictionary Learningcited by 67
  15. 2024The Geometry of Concepts: Sparse Autoencoder Feature Structurecited by 52
  16. 2024Decomposing The Dark Matter of Sparse Autoencoderscited by 45
  17. 2024Efficient Dictionary Learning with Switch Sparse Autoencoderscited by 39
  18. 2024Evaluating Open-Source Sparse Autoencoders on Disentangling Factual Knowledge in GPT-2 Smallcited by 34
  19. 2025Automated Interpretability Metrics Do Not Distinguish Trained and Random Transformerscited by 30
  20. 2024Monotonic Representation of Numeric Properties in Language Modelscited by 15
  21. 2024Features that Make a Difference: Leveraging Gradients for Improved Dictionary Learningcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 43; 2026: 72.

115 papers.

Sort:

By area:

  1. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelscited by 76
  2. 2025Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometrycited by 61
  3. 2025Detecting High-Stakes Interactions with Activation Probescited by 53
  4. 2025Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainerscited by 52
  5. 2026Building Production-Ready Probes For Geminicited by 47
  6. 2026Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreakscited by 41
  7. 2025Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuningcited by 39
  8. 2026Do Sparse Autoencoders Capture Concept Manifolds?cited by 31
  9. 2025SAE-SSV: Supervised Steering in Sparse Representation Spaces for Reliable Control of Language Modelscited by 29
  10. 2025To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Modelscited by 29
  11. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 27
  12. 2026Language Model Circuits Are Sparse in the Neuron Basiscited by 25
38 more
  1. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 24
  2. 2025RL-Obfuscation: Can Language Models Learn to Evade Latent-Space Monitors?cited by 24
  3. 2025Transferring Linear Features Across Language Models With Model Stitchingcited by 21
  4. 2025Position: Use Sparse Autoencoders to Discover Unknownscited by 19
  5. 2025Temporal Sparse Autoencoders: Leveraging the Sequential Nature of Language for Interpretabilitycited by 18
  6. 2025Feature Hedging: Correlated Features Break Narrow Sparse Autoencoderscited by 16
  7. 2025Investigating task-specific prompts and sparse autoencoders for activation monitoringcited by 15
  8. 2025Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoderscited by 14
  9. 2026Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?cited by 14
  10. 2025I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?cited by 13
  11. 2026Shaping capabilities with token-level data filteringcited by 13
  12. 2025The Dead Salmons of AI Interpretabilitycited by 12
  13. 2025Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?cited by 11
  14. 2026When Benchmarks Lie: Evaluating Malicious Prompt Classifiers Under True Distribution Shiftcited by 11
  15. 2026SynthSAEBench: Evaluating Sparse Autoencoders on Scalable Realistic Synthetic Datacited by 11
  16. 2025Probing the Representational Power of Sparse Autoencoders in Vision Modelscited by 10
  17. 2025AbsTopK: Rethinking Sparse Autoencoders For Bidirectional Featurescited by 9
  18. 2025Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Unseen Activation Monitorscited by 9
  19. 2025Do Sparse Autoencoders Generalize? A Case Study of Answerabilitycited by 8
  20. 2025Can Interpretation Predict Behavior on Unseen Data?cited by 8
  21. 2025Hypothesis-Driven Feature Manifold Analysis in LLMs via Supervised Multi-Dimensional Scalingcited by 8
  22. 2026Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetrycited by 8
  23. 2026AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoderscited by 8
  24. 2026Learning a Generative Meta-Model of LLM Activationscited by 8
  25. 2026Causality is Key for Interpretability Claims to Generalisecited by 8
  26. 2026The Expert Strikes Back: Interpreting Mixture-of-Experts Language Models at Expert Levelcited by 7
  27. 2025Rethinking Explainability in the Era of Multimodal AIcited by 6
  28. 2025Interpretable and Steerable Concept Bottleneck Sparse Autoencoderscited by 6
  29. 2026Sparse Autoencoders are Capable LLM Jailbreak Mitigatorscited by 6
  30. 2026Learning Retrieval Models with Sparse Autoencoderscited by 6
  31. 2026Stop Probing, Start Coding: Why Linear Probes and Sparse Autoencoders Fail at Compositional Generalisationcited by 6
  32. 2025Towards Interpretability Without Sacrifice: Faithful Dense Layer Decomposition with Mixture of Decoderscited by 5
  33. 2026Stable and Steerable Sparse Autoencoders with Weight Regularizationcited by 5
  34. 2026Monitoring the Internal Monologue: Probe Trajectories Reveal Reasoning Dynamicscited by 5
  35. 2025Evaluating Sparse Autoencoders for Monosemantic Representationcited by 4
  36. 2026TADA! Tuning Audio Diffusion Models through Activation Steeringcited by 4
  37. 2026Linear Probe Accuracy Scales with Model Size and Benefits from Multi-Layer Ensemblingcited by 4
  38. 2026Are Sparse Autoencoder Benchmarks Reliable?cited by 4

50 most cited of 115 All 115 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Sparse Autoencoders Trained on the Same Data Learn Different Features
  2. 2025Sparse Shift Autoencoders for Identifying Concepts from Large Language Model Activations
  3. 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
  4. 2024Evaluating Sparse Autoencoders on Targeted Concept Erasure Tasks
  5. 2025Beyond Input Activations: Identifying Influential Latents by Gradient Sparse Autoencoders
  6. 2025Evaluating SAE interpretability without explanations
  7. 2024Automatically Interpreting Millions of Features in Large Language Models
  8. 2025Unveiling Decision-Making in LLMs for Text Classification : Extraction of influential and interpretable concepts with Sparse Autoencoders