# Graphical Design of Interpretable Architectures

arXiv: https://arxiv.org/abs/2608.18936 · HTML: https://mlsanity.com/paper/2608.18936/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 35 references that are in the map.
- First posted on arXiv 19 Aug 2026.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [Neural Additive Models: Interpretable Machine Learning with Neural Nets](https://mlsanity.com/paper/2004.13912/) (2020)
- [Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet](https://mlsanity.com/paper/2605.29358/) (2026)
- [Interpretable Neural-Symbolic Concept Reasoning](https://mlsanity.com/paper/2304.14068/) (2023)
- [Categorical Foundations of Gradient-Based Learning](https://mlsanity.com/paper/2103.01931/) (2021)
- [Position: Categorical Deep Learning is an Algebraic Theory of All Architectures](https://mlsanity.com/paper/2402.15332/) (2024)
- [Interpretable Concept-Based Memory Reasoning](https://mlsanity.com/paper/2407.15527/) (2024)
- [Causal models in string diagrams](https://mlsanity.com/paper/2304.07638/) (2023)
- [Towards Compositional Interpretability for XAI](https://mlsanity.com/paper/2406.17583/) (2024)
- [Named Tensor Notation](https://mlsanity.com/paper/2102.13196/) (2021)
- [An introduction to graphical tensor notation for mechanistic interpretability](https://mlsanity.com/paper/2402.01790/) (2024)
- [Neural Circuit Diagrams: Robust Diagrams for the Communication, Implementation, and Analysis of Deep Learning Architectures](https://mlsanity.com/paper/2402.05424/) (2024)
- [Categorical Foundations of Explainable AI: A Unifying Theory](https://mlsanity.com/paper/2304.14094/) (2023)
- [FlashAttention on a Napkin: A Diagrammatic Approach to Deep Learning IO-Awareness](https://mlsanity.com/paper/2412.03317/) (2024)
- [Foundations of Interpretable Models](https://mlsanity.com/paper/2508.00545/) (2025)
- [Mixture of Concept Bottleneck Experts](https://mlsanity.com/paper/2602.02886/) (2026)
- [Prototype-Grounded Concept Models for Verifiable Concept Alignment](https://mlsanity.com/paper/2604.16076/) (2026)
- [The Standard Interpretable Model: A general theory of interpretable machine learning to deductively design interpretable methods using Lagrangian mechanics](https://mlsanity.com/paper/2606.12289/) (2026)
- [Scaling Inherently Interpretable Language Models](https://mlsanity.com/paper/2608.07594/) (2026)

## Abstract (arXiv)

Designing, implementing, and comparing interpretable architectures requires a formal language to represent them. The most common representations fall short in one of two ways. Symbolic equations give no global view of an architecture at a glance. Probabilistic graphical models and flowcharts do not describe actual tensor manipulations, thus hiding key insights and limiting reproducibility. To close this gap, we introduce a graphical notation for designing interpretable AI architectures, adapted from Penrose tensor notation. This graphical notation gives a global view of an architecture and maps one to one onto PyTorch einsum code. We first use this notation to describe architectures that are interpretable by construction, including concept bottlenecks, sparse probes, prototype networks, neural additive models, and mixtures of linear models. We then diagram the key architectural components of Steerling-8B, a frontier interpretable language model. The diagram yields global insights into the architecture (e.g., showing that Steerling is a residual model), a geometric interpretation of each individual operation, and a direct translation into 33 lines of PyTorch code.