mlsanityData to 7 Oct 2026

Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Models

arXiv 2408.00113 ↗First posted 31 Jul 2024cs.LG cs.AI cs.CLcited by 51

Not on the map. Why?

Abstract (arXiv)

What latent features are encoded in language model (LM) representations? Recent work on training sparse autoencoders (SAEs) to disentangle interpretable features in LM representations has shown significant promise. However, evaluating the quality of these SAEs is difficult because we lack a ground-truth collection of interpretable features that we expect good SAEs to recover. We thus propose to measure progress in interpretable dictionary learning by working in the setting of LMs trained on chess and Othello transcripts. These settings carry natural collections of interpretable features -- for example, "there is a knight on F3" -- which we leverage into textitsupervised metrics for SAE quality. To guide progress in interpretable dictionary learning, we introduce a new SAE training technique, textitp-annealing, which improves performance on prior unsupervised metrics as well as our new metrics.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). Builds on 22 of its 65 references that are in the map. First posted on arXiv 31 Jul 2024.

Builds on

22 of 65 references.

References · 22

  1. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  2. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  3. 2022Toy Models of Superpositioncited by 928
  4. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  5. 2024Scaling and evaluating sparse autoencoderscited by 530
  6. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  7. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  8. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  9. 2013k-Sparse Autoencoderscited by 209
  10. 2024Improving Dictionary Learning with Gated Sparse Autoencoderscited by 171
  11. 2016Linear Algebraic Structure of Word Senses, with Applications to Polysemycited by 165
  12. 2021Acquisition of Chess Knowledge in AlphaZerocited by 120
10 more
  1. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  2. 2024Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Controlcited by 83
  3. 2024Emergent World Models and Latent Variable Estimation in Chess-Playing Language Modelscited by 60
  4. 2024Interpreting Attention Layer Outputs with Sparse Autoencoderscited by 43
  5. 2015Simple, Efficient, and Neural Algorithms for Sparse Codingcited by 37
  6. 2014On the Computational Intractability of Exact and Approximate Dictionary Learningcited by 14
  7. 2015Why Regularized Auto-Encoders learn Sparse Representation?cited by 11
  8. 2017Sparse Coding and Autoencoderscited by 11
  9. 2014Performance Limits of Dictionary Learning for Sparse Codingcited by 2
  10. 2016An improved analysis of the ER-SpUD dictionary learning algorithmcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 25; 2026: 15.

51 papers.

Sort:

By area:

  1. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
  2. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 148
  3. 2025Learning Multi-Level Features with Matryoshka Sparse Autoencoderscited by 136
  4. 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilitycited by 118
  5. 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probingcited by 109
  6. 2025Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Modelscited by 67
  7. 2024InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoderscited by 64
  8. 2025Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languagescited by 57
  9. 2024Decomposing The Dark Matter of Sparse Autoencoderscited by 42
  10. 2024Towards Unifying Interpretability and Control: Evaluation via Interventioncited by 28
  11. 2024Mechanistic?cited by 25
  12. 2026Do Sparse Autoencoders Capture Concept Manifolds?cited by 25
38 more
  1. 2024The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysiscited by 24
  2. 2025Mixture of Experts Made Intrinsically Interpretablecited by 22
  3. 2025Temporal Sparse Autoencoders: Leveraging the Sequential Nature of Language for Interpretabilitycited by 16
  4. 2025Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEscited by 14
  5. 2024Transformers Use Causal World Models in Maze-Solving Taskscited by 13
  6. 2025I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?cited by 13
  7. 2025Evaluating Adversarial Robustness of Concept Representations in Sparse Autoencoderscited by 11
  8. 2026Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?cited by 10
  9. 2024Adaptive Sparse Allocation with Mutual Choice & Feature Choice Sparse Autoencoderscited by 9
  10. 2025Does higher interpretability imply better utility? A Pairwise Analysis on Sparse Autoencoderscited by 9
  11. 2025Crosscoding Through Time: Tracking Emergence & Consolidation Of Linguistic Representations Throughout LLM Pretrainingcited by 6
  12. 2026From Atoms to Trees: Building a Structured Feature Forest with Hierarchical Sparse Autoencoderscited by 6
  13. 2024Features that Make a Difference: Leveraging Gradients for Improved Dictionary Learningcited by 5
  14. 2025LLM Interpretability with Identifiable Temporal-Instantaneous Representationcited by 4
  15. 2025Measuring Sparse Autoencoder Feature Sensitivitycited by 4
  16. 2025Revisiting End-To-End Sparse Autoencoder Training: A Short Finetune Is All You Needcited by 3
  17. 2025Inference-Time Decomposition of Activations (ITDA): A Scalable Approach to Interpreting Large Language Modelscited by 3
  18. 2025AdaptiveK: Complexity-Driven Sparse Autoencoders for Interpretable Language Model Representationscited by 3
  19. 2026Are Sparse Autoencoder Benchmarks Reliable?cited by 3
  20. 2024SAGE: Scalable Ground Truth Evaluations for Large Sparse Autoencoderscited by 2
  21. 2025CE-Bench: Towards a Reliable Contrastive Evaluation Benchmark of Interpretability of Sparse Autoencoderscited by 2
  22. 2025A Unified Theory of Sparse Dictionary Learning in Mechanistic Interpretability: Piecewise Biconvexity and Spurious Minimacited by 2
  23. 2026Tracing the Thought of a Grandmaster-level Chess-Playing Transformercited by 2
  24. 2026Improving Sparse Autoencoder with Dynamic Attentioncited by 2
  25. 2025Cognitive Activation and Chaotic Dynamics in Large Language Models: A Quasi-Lyapunov Analysis of Reasoning Mechanismscited by 1
  26. 2025ConceptViz: A Visual Analytics Approach for Exploring Concepts in Large Language Modelscited by 1
  27. 2026Aligned Training: A Parameter-Free Method to Improve Feature Quality and Stability of Sparse Autoencoders (SAE)cited by 1
  28. 2026ReSAE: Residualized Sparse Autoencoders for Multi-Layer Transformer Interventionscited by 1
  29. 2025Dictionary Learning: The Complexity of Learning Sparse Superposed Features with Feedbackcited by 0
  30. 2025Sparsification and Reconstruction from the Perspective of Representation Geometrycited by 0
  31. 2025What if Othello-Playing Language Models Could See?cited by 0
  32. 2025Automatically Finding Rule-Based Neurons in OthelloGPTcited by 0
  33. 2026Concept Component Analysis: A Principled Approach for Concept Extraction in LLMscited by 0
  34. 2026Sign-Aware Gated Sparse Autoencoders: Modeling Anticorrelated Features with Bi-Jump-ReLU Activationscited by 0
  35. 2026Cascaded Sparse Autoencoders Learn Multi-Level Visual Concepts in Multimodal LLMscited by 0
  36. 2026Where You Measure Decides What You Measure: Position Selection in Ablation-Based SAE Evaluationcited by 0
  37. 2026A Unifying Perspective on Language Model Representations: From Filler-Role Structure to Mechanistic Interpretabilitycited by 0
  38. 2026A Survey on the Linear Representation Hypothesiscited by 0

50 most cited of 51 All 51 (JSON)