Representation as a Bottleneck for Mechanistic Interpretability: The Manifestation Unit Protocol
On the map: Mechanistic interpretability → Circuits and mechanistic analysis
Abstract (arXiv)
Mechanistic interpretability has produced a rich inventory of component-level analyses that characterise what neural-network components encode and how they interact. Their outputs, however, are not easily reusable: selectivity tables, circuit diagrams, and feature lists remain locked in per-study notebooks - non-composable, not queryable in natural language, and not directly actionable for downstream audit or intervention. We study the representation layer that sits between these analyses and downstream use as a bottleneck that can be evaluated independently, and introduce Manifestation Units, a typed tuple protocol (E, S, R, D, G) extended with attention-head primitives (T) for transformer architectures, organising per-component statistics into structured fields populated automatically and queried through hybrid retrieval. Instantiated across generative vision (beta-VAE), discriminative vision (CNN), and language (GPT-2), the protocol supports two findings: typed structure substantially outperforms unstructured baselines on retrieval, and CNN filters retrieved by the schema satisfy causal sufficiency and necessity criteria under matched-budget controls. The schema absorbs attention-head primitives without modification, set-recovers known IOI circuit members under retrieval-budget-matched controls, and reveals an irreducible two-field core (S+R) with remaining fields either redundant or actively interfering. We present this as schema infrastructure for mechanistic interpretability rather than frontier-scale validation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability. Builds on 5 of its 30 references that are in the map. First posted on arXiv 30 Jun 2026.
Builds on
5 of 30 references.
Closest ancestors · 1
Same direction, newest first
Other references · 4
- 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
- 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
- 2024How to use and interpret activation patchingcited by 214
- 2023Localizing Model Behavior with Path Patchingcited by 195
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIP
- 2026MechaTerp-TRACE: A Novel Approach for Component Ablation Analysis in Language Models
- 2026From Mechanistic to Compositional Interpretability
- 2023Scale Alone Does not Improve Mechanistic Interpretability in Vision Models
- 2026Mechanistically Interpretable Neural Encoding Reveals Fine-Grained Functional Selectivity in Human Visual Cortex
- 2026Causally Grounded Mechanistic Interpretability for LLMs with Faithful Natural-Language Explanations
- 2026Sparse Transformer: a Controlled Testbed for Mechanistic Interpretability
- 2024The Cognitive Revolution in Interpretability: From Explaining Behavior to Interpreting Representations and Algorithms