mlsanityData to 14 Oct 2026

Sparse Transformer: a Controlled Testbed for Mechanistic Interpretability

arXiv 2610.19907 ↗First posted 14 Oct 2026cs.AIcited by 0

On the map: Mechanistic interpretability → Sparse autoencoder training and evaluation also fits: Circuits and mechanistic analysis

Abstract (arXiv)

Mechanistic interpretability aims to decompose neural network computations into human-understandable features, yet progress is limited by the absence of ground-truth units to evaluate against. We address this with a Sparse Transformer (ST): a 5-layer language model trained on TinyStories whose wide, TopK-sparse MLP forces neuron specialization. We show that ST neurons are highly interpretable: they outperform a loss-matched dense transformer by 43% on AutoInterp, and circuits of a median of 12 neurons suffice to reproduce individual model predictions. This testbed allows us to directly compare feature learning methods: Transcoder, which approximates the MLP input-output mapping, recovers ST's neuron vectors with 52.6% higher cosine similarity than Sparse Autoencoder, which only reconstructs outputs. Furthermore, the ST enables a controlled study of polysemanticity: distilling its sparse MLP into a dense counterpart reveals how compression leads to feature superposition. These results show how a known reference basis sharpens evaluation and understanding, a role we hope the ST can serve for the field.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Sparse autoencoder training and evaluation, one of 13 directions in Mechanistic interpretability. Builds on 32 of its 44 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

32 of 44 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026SynthSAEBench: Evaluating Sparse Autoencoders on Scalable Realistic Synthetic Data16 Feb
  2. 2026Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?15 Feb
  3. 2025Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders22 Aug
  4. 2025Learning Multi-Level Features with Matryoshka Sparse Autoencoders21 Mar
  5. 2025Sparse Autoencoders Do Not Find Canonical Units of Analysis7 Feb
  6. 2025Transcoders Beat Sparse Autoencoders for Interpretability31 Jan
  7. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 29 Aug

Other references · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Scaling Laws for Neural Language Modelscited by 6,586
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  6. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  7. 2019Root Mean Square Layer Normalizationcited by 1,480
  8. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  9. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  10. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  11. 2020A Primer in BERTology: What we know about how BERT workscited by 815
  12. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
13 more
  1. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 449
  2. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  3. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
  4. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 300
  5. 2024Transcoders Find Interpretable LLM Feature Circuitscited by 200
  6. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 160
  7. 2024LLM Circuit Analyses Are Consistent Across Training and Scalecited by 63
  8. 2019Explicit Sparse Transformer: Concentrated Attention Through Explicit Selectioncited by 62
  9. 2025Weight-sparse transformers have interpretable circuitscited by 44
  10. 2025Advancing Expert Specialization for Better MoEcited by 39
  11. 2025The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?cited by 37
  12. 2026Language Model Circuits Are Sparse in the Neuron Basiscited by 25
  13. 2025Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Frameworkcited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Mechanistic Interpretability for Neural Networks: Circuits, Sparse Features and Symbolic Reasoning
  2. 2026Sparse but not Simpler: A Multi-Level Interpretability Analysis of Vision Transformers
  3. 2024Improving Neuron-level Interpretability with White-box Language Models
  4. 2025A Unified Theory of Sparse Dictionary Learning in Mechanistic Interpretability: Piecewise Biconvexity and Spurious Minima
  5. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probing
  6. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models
  7. 2024Interpreting Attention Layer Outputs with Sparse Autoencoders
  8. 2026Sparsely gated tiny linear experts