# Evaluating SAE interpretability without explanations

arXiv: https://arxiv.org/abs/2507.08473 · HTML: https://mlsanity.com/paper/2507.08473/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in SAE features and their applications, one of 13 directions in Mechanistic interpretability.
- Builds on 17 of its 28 references that are in the map.
- First posted on arXiv 11 Jul 2025.

## Placed in

- Mechanistic interpretability / [SAE features and their applications](https://mlsanity.com/mechinterp/sae-features-and-their-applications/)

## Builds on (most cited first, 17 of 17)

- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Scaling and evaluating sparse autoencoders](https://mlsanity.com/paper/2406.04093/) (2024)
- [Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2](https://mlsanity.com/paper/2408.05147/) (2024)
- [SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model](https://mlsanity.com/paper/2502.02737/) (2025)
- [Compositional Explanations of Neurons](https://mlsanity.com/paper/2006.14032/) (2020)
- [A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders](https://mlsanity.com/paper/2409.14507/) (2024)
- [SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability](https://mlsanity.com/paper/2503.09532/) (2025)
- [Automatically Interpreting Millions of Features in Large Language Models](https://mlsanity.com/paper/2410.13928/) (2024)
- [SPINE: SParse Interpretable Neural Embeddings](https://mlsanity.com/paper/1711.08792/) (2017)
- [Enhancing Automated Interpretability with Output-Centric Feature Descriptions](https://mlsanity.com/paper/2501.08319/) (2025)
- [The Missing Curve Detectors of InceptionV1: Applying Sparse Autoencoders to InceptionV1 Early Vision](https://mlsanity.com/paper/2406.03662/) (2024)
- [Interpretability as Compression: Reconsidering SAE Explanations of Neural Activations with MDL-SAEs](https://mlsanity.com/paper/2410.11179/) (2024)
- [Scale Alone Does not Improve Mechanistic Interpretability in Vision Models](https://mlsanity.com/paper/2307.05471/) (2023)
- [From superposition to sparse codes: interpretable representations in neural networks](https://mlsanity.com/paper/2503.01824/) (2025)
- [Learning biologically relevant features in a pathology foundation model using sparse autoencoders](https://mlsanity.com/paper/2407.10785/) (2024)
- [Partially Rewriting a Transformer in Natural Language](https://mlsanity.com/paper/2501.18838/) (2025)

## Built on by (most cited first, 5 of 5)

- [ADAPT: Hybrid Prompt Optimization for LLM Feature Visualization](https://mlsanity.com/paper/2602.17867/) (2026)
- [Sign-Aware Gated Sparse Autoencoders: Modeling Anticorrelated Features with Bi-Jump-ReLU Activations](https://mlsanity.com/paper/2605.28149/) (2026)
- [Efficient Auto-Interpretability of AI Models in Biology](https://mlsanity.com/paper/2608.27754/) (2026)
- [SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research?](https://mlsanity.com/paper/2609.09113/) (2026)
- [Weights Read and Write Features: Scalable Parameter Decomposition Grounded in Activation Space](https://mlsanity.com/paper/2609.37731/) (2026)

## Abstract (arXiv)

Sparse autoencoders (SAEs) and transcoders have become important tools for machine learning interpretability. However, measuring how interpretable they are remains challenging, with weak consensus about which benchmarks to use. Most evaluation procedures start by producing a single-sentence explanation for each latent. These explanations are then evaluated based on how well they enable an LLM to predict the activation of a latent in new contexts. This method makes it difficult to disentangle the explanation generation and evaluation process from the actual interpretability of the latents discovered. In this work, we adapt existing methods to assess the interpretability of sparse coders, with the advantage that they do not require generating natural language explanations as an intermediate step. This enables a more direct and potentially standardized assessment of interpretability. Furthermore, we compare the scores produced by our interpretability metrics with human evaluations across similar tasks and varying setups, offering suggestions for the community on improving the evaluation of these techniques.