mlsanityData to 7 Oct 2026

Towards Interpretability Without Sacrifice: Faithful Dense Layer Decomposition with Mixture of Decoders

arXiv 2505.21364 ↗First posted 27 May 2025cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Multilayer perceptrons (MLPs) are an integral part of large language models, yet their dense representations render them difficult to understand, edit, and steer. Recent methods learn interpretable approximations via neuron-level sparsity, yet fail to faithfully reconstruct the original mapping--significantly increasing model's next-token cross-entropy loss. In this paper, we advocate for moving to layer-level sparsity to overcome the accuracy trade-off in sparse layer approximation. Under this paradigm, we introduce Mixture of Decoders (MxDs). MxDs generalize MLPs and Gated Linear Units, expanding pre-trained dense layers into tens of thousands of specialized sublayers. Through a flexible form of tensor factorization, each sparsely activating MxD sublayer implements a linear transformation with full-rank weights--preserving the original decoders' expressive capacity even under heavy sparsity. Experimentally, we show that MxDs significantly outperform state-of-the-art methods (e.g., Transcoders) on the sparsity-accuracy frontier in language models with up to 3B parameters. Further evaluations on sparse probing and feature steering demonstrate that MxDs learn similarly specialized features of natural language--opening up a promising new avenue for designing interpretable yet faithful decompositions. Our code is included at: https://github.com/james-oldfield/MxD/.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 64 of its 104 references that are in the map. First posted on arXiv 27 May 2025.

Builds on

64 of 104 references.

References · 64

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  7. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  8. 2020GLU Variants Improve Transformercited by 1,724
  9. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  10. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  11. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  12. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
38 more
  1. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  2. 2021Scaling Vision with Sparse Mixture of Expertscited by 740
  3. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  4. 2016The Mythos of Model Interpretabilitycited by 716
  5. 2024Scaling and evaluating sparse autoencoderscited by 530
  6. 2021Dynamic Neural Networks: A Surveycited by 405
  7. 2022ST-MoE: Designing Stable and Transferable Sparse Expert Modelscited by 391
  8. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
  9. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  10. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 283
  11. 2019CondConv: Conditionally Parameterized Convolutions for Efficient Inferencecited by 273
  12. 2018Manipulating and Measuring Model Interpretabilitycited by 256
  13. 2023From Sparse to Soft Mixtures of Expertscited by 245
  14. 2024LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Taskscited by 236
  15. 2024Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoderscited by 213
  16. 2013k-Sparse Autoencoderscited by 209
  17. 2022Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Expertscited by 202
  18. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  19. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 197
  20. 2024Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Expertscited by 191
  21. 2024Transcoders Find Interpretable LLM Feature Circuitscited by 186
  22. 2025Open Problems in Mechanistic Interpretabilitycited by 173
  23. 2015Conditional Computation in Neural Networks for faster modelscited by 166
  24. 2016Linear Algebraic Structure of Word Senses, with Applications to Polysemycited by 165
  25. 2022On the Representation Collapse of Sparse Mixture of Expertscited by 156
  26. 2023Pushing Mixture of Experts to the Limit: Extremely Parameter Efficient MoE for Instruction Tuningcited by 148
  27. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 148
  28. 2024BatchTopK Sparse Autoencoderscited by 126
  29. 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilitycited by 118
  30. 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probingcited by 109
  31. 2023Interpretability at Scale: Identifying Causal Mechanisms in Alpacacited by 102
  32. 2025Sparse Autoencoders Trained on the Same Data Learn Different Featurescited by 86
  33. 2025Sparse Autoencoders Do Not Find Canonical Units of Analysiscited by 80
  34. 2024Mixture of A Million Expertscited by 70
  35. 2024Applying sparse autoencoders to unlearn knowledge in language modelscited by 58
  36. 2022Overlooked factors in concept-based explanations: Dataset choice, concept learnability, and human capabilitycited by 46
  37. 2024Decomposing The Dark Matter of Sparse Autoencoderscited by 42
  38. 2023Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizercited by 28

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026PolySAE: Modeling Feature Interactions in Sparse Autoencoders via Polynomial Decodingcited by 3
  2. 2026Towards Intrinsic Interpretability of Large Language Models:A Survey of Design Principles and Architecturescited by 3
  3. 2026Toward Structural Multimodal Representations: Specialization, Selection, and Sparsification via Mixture-of-Expertscited by 0
  4. 2026fmxcoders: Factorized Masked Crosscoders for Cross-Layer Feature Discoverycited by 0
  5. 2026Faithful by Definition: Emotion Analysis via Natural Semantic Metalanguage Explicationscited by 0