Sparse Readout Prism: Explaining Logit-Lens Scores in Features Instead of Tokens
Not on the map. Why?
Abstract (arXiv)
A language model's prediction of its next token develops across layers, and lens methods track this process by decoding intermediate hidden states into tokens. But a lens reading reflects both the hidden state and the readout (the unembedding matrix) used to decode it. Many lenses are fit on a corpus, and we show that two lenses differing only in their fitting corpus can report different tokens for the same hidden states. We call this dependence corpus conditionality. To examine readout structure independently of the fitting corpus, we introduce Sparse Readout Prism (SRP), which decomposes the readout using only its weights and expresses any token logit or logit difference as a sum of contributions from sparse readout features. This reveals readout features as a new unit of analysis for lens readings, exposing structure that token identities can obscure and enabling comparisons across tokens, contexts, layers, and lenses. Replacing the original readout with SRP's sparse approximation reconstructs 8.9-17.3 percentage points more of the tested logit differences than the strongest of six baselines built on geometric relations among readout rows. Ablating features shifts logit differences in proportion to their SRP contributions. Although token readings vary with the fitting corpus, the dominant readout feature remains stable. Because SRP uses no corpus in its construction, it provides a control independent of the fitting corpus for lens analyses.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 36 of its 80 references that are in the map. First posted on arXiv 1 Sep 2026.
Builds on
36 of 80 references.
References · 36
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2025Qwen3 Technical Reportcited by 8,820
- 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
- 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
- 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
- 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
- 2024Scaling and evaluating sparse autoencoderscited by 553
- 2023LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Modelscited by 336
- 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 300
24 more
- 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 235
- 2013k-Sparse Autoencoderscited by 215
- 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
- 2017Breaking the Softmax Bottleneck: A High-Rank RNN Language Modelcited by 152
- 2025Learning Multi-Level Features with Matryoshka Sparse Autoencoderscited by 143
- 2024BatchTopK Sparse Autoencoderscited by 130
- 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilitycited by 124
- 2026Ministral 3cited by 114
- 2025Sparse Autoencoders Trained on the Same Data Learn Different Featurescited by 93
- 2016Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modelingcited by 92
- 2025Sparse Autoencoders Do Not Find Canonical Units of Analysiscited by 88
- 2024Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Controlcited by 86
- 2025Do Multilingual LLMs Think In English?cited by 78
- 2025Weight-sparse transformers have interpretable circuitscited by 44
- 2025LLM Unlearning via Neural Activation Redirectioncited by 41
- 2025Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat-Tuningcited by 34
- 2025Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decompositioncited by 27
- 2024Disentangling Dense Embeddings with Sparse Autoencoderscited by 23
- 2025LogitLens4LLMs: Extending Logit Lens Analysis to Modern Large Language Modelscited by 23
- 2026Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Modelscited by 21
- 2026LatentLens: Revealing Highly Interpretable Visual Tokens in LLMscited by 15
- 2024DEPT: Decoupled Embeddings for Pre-training Language Modelscited by 5
- 2026Output Embedding Centering for Stable LLM Pretrainingcited by 4
- 2026Beyond Activation Patterns: A Weight-Based Out-of-Context Explanation of Sparse Autoencoder Featurescited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.