# Bilinear MLPs enable weight-based mechanistic interpretability

arXiv: https://arxiv.org/abs/2410.08417 · HTML: https://mlsanity.com/paper/2410.08417/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability.
- Builds on 23 of its 37 references that are in the map.
- First posted on arXiv 10 Oct 2024.

## Placed in

- Mechanistic interpretability / [Circuits and mechanistic analysis](https://mlsanity.com/mechinterp/circuits-and-mechanistic-analysis/)

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Language Modeling with Gated Convolutional Networks](https://mlsanity.com/paper/1612.08083/) (2016)
- [Scaling and evaluating sparse autoencoders](https://mlsanity.com/paper/2406.04093/) (2024)
- [TinyStories: How Small Can Language Models Be and Still Speak Coherent English?](https://mlsanity.com/paper/2305.07759/) (2023)
- [Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models](https://mlsanity.com/paper/2403.19647/) (2024)
- [Methods for Interpreting and Understanding Deep Neural Networks](https://mlsanity.com/paper/1706.07979/) (2017)
- [Bilinear CNNs for Fine-grained Visual Recognition](https://mlsanity.com/paper/1504.07889/) (2015)
- [Tensor Decomposition for Signal Processing and Machine Learning](https://mlsanity.com/paper/1607.01668/) (2016)
- [Convergent Learning: Do different neural networks learn the same representations?](https://mlsanity.com/paper/1511.07543/) (2015)
- [Deep Polynomial Neural Networks](https://mlsanity.com/paper/2006.13026/) (2020)
- [Tensor Methods in Computer Vision and Deep Learning](https://mlsanity.com/paper/2107.03436/) (2021)
- [Automatically Identifying Local and Global Circuits with Linear Computation Graphs](https://mlsanity.com/paper/2405.13868/) (2024)
- [Factorized Bilinear Models for Image Recognition](https://mlsanity.com/paper/1611.05709/) (2016)
- [A technical note on bilinear layers for interpretability](https://mlsanity.com/paper/2305.03452/) (2023)

## Built on by (most cited first, 28 of 28)

- [Language Model Circuits Are Sparse in the Neuron Basis](https://mlsanity.com/paper/2601.22594/) (2026)
- [Mixture of Experts Made Intrinsically Interpretable](https://mlsanity.com/paper/2503.07639/) (2025)
- [COSMIC: Generalized Refusal Direction Identification in LLM Activations](https://mlsanity.com/paper/2506.00085/) (2025)
- [Beyond Linear Probes: Dynamic Safety Monitoring for Language Models](https://mlsanity.com/paper/2509.26238/) (2025)
- [Parameterized Synthetic Text Generation with SimpleStories](https://mlsanity.com/paper/2504.09184/) (2025)
- [Compositionality Unlocks Deep Interpretable Models](https://mlsanity.com/paper/2504.02667/) (2025)
- [Towards Interpretability Without Sacrifice: Faithful Dense Layer Decomposition with Mixture of Decoders](https://mlsanity.com/paper/2505.21364/) (2025)
- [Finding Manifolds With Bilinear Autoencoders](https://mlsanity.com/paper/2510.16820/) (2025)
- [Towards Intrinsic Interpretability of Large Language Models:A Survey of Design Principles and Architectures](https://mlsanity.com/paper/2604.16042/) (2026)
- [PolySAE: Modeling Feature Interactions in Sparse Autoencoders via Polynomial Decoding](https://mlsanity.com/paper/2602.01322/) (2026)
- [Disentangling MLP Neuron Weights in Vocabulary Space](https://mlsanity.com/paper/2604.06005/) (2026)
- [Explicit Fuzzy Logic in the Feed-Forward Layer: Self-Forgetting Quantifiers Discover Legible Grammatical-Licensing Detectors](https://mlsanity.com/paper/2606.31845/) (2026)
- [Legible-by-Construction: Attention and End-to-End Transformers](https://mlsanity.com/paper/2607.04319/) (2026)
- [Task complexity shapes internal representations and robustness in neural networks](https://mlsanity.com/paper/2508.05463/) (2025)
- [Hedonic Neurons: A Mechanistic Mapping of Latent Coalitions in Transformer MLPs](https://mlsanity.com/paper/2509.23684/) (2025)
- [Bilinear autoencoders find interpretable manifolds](https://mlsanity.com/paper/2605.08891/) (2026)
- [Training, Reading, and Editing Legible Transformers](https://mlsanity.com/paper/2607.08946/) (2026)
- [Bilinear Convolution Decomposition for Causal RL Interpretability](https://mlsanity.com/paper/2412.00944/) (2024)
- [Preserving Bilinear Weight Spectra with a Signed and Shrunk Quadratic Activation Function](https://mlsanity.com/paper/2509.01874/) (2025)
- [Structural Disentanglement in Bilinear MLPs via Architectural Inductive Bias](https://mlsanity.com/paper/2602.05635/) (2026)
- [Private and interpretable clinical prediction with quantum-inspired tensor train models](https://mlsanity.com/paper/2602.06110/) (2026)
- [Half the Nonlinearity Is Wasted: Measuring and Reallocating the Transformer's MLP Budget](https://mlsanity.com/paper/2603.03459/) (2026)
- [On the Role of Depth in the Expressivity of RNNs](https://mlsanity.com/paper/2604.02201/) (2026)
- [Sparsity Moves Computation: How FFN Architecture Reshapes Attention in Small Transformers](https://mlsanity.com/paper/2605.09403/) (2026)
- [When Are Two Networks the Same? Tensor Similarity for Mechanistic Interpretability](https://mlsanity.com/paper/2605.15183/) (2026)
- [Individual Parameters in Weight-Sparse Transformers Appear Interpretable](https://mlsanity.com/paper/2607.02964/) (2026)
- [Benign Loss Landscapes Can Coexist with Worst-Case Hardness](https://mlsanity.com/paper/2609.13057/) (2026)
- [Weights Read and Write Features: Scalable Parameter Decomposition Grounded in Activation Space](https://mlsanity.com/paper/2609.37731/) (2026)

## Abstract (arXiv)

A mechanistic understanding of how MLPs do computation in deep neural networks remains elusive. Current interpretability work can extract features from hidden activations over an input dataset but generally cannot explain how MLP weights construct features. One challenge is that element-wise nonlinearities introduce higher-order interactions and make it difficult to trace computations through the MLP layer. In this paper, we analyze bilinear MLPs, a type of Gated Linear Unit (GLU) without any element-wise nonlinearity that nevertheless achieves competitive performance. Bilinear MLPs can be fully expressed in terms of linear operations using a third-order tensor, allowing flexible analysis of the weights. Analyzing the spectra of bilinear MLP weights using eigendecomposition reveals interpretable low-rank structure across toy tasks, image classification, and language modeling. We use this understanding to craft adversarial examples, uncover overfitting, and identify small language model circuits directly from the weights alone. Our results demonstrate that bilinear layers serve as an interpretable drop-in replacement for current activation functions and that weight-based interpretability is viable for understanding deep-learning models.