# Scale Alone Does not Improve Mechanistic Interpretability in Vision Models

arXiv: https://arxiv.org/abs/2307.05471 · HTML: https://mlsanity.com/paper/2307.05471/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability.
- Builds on 39 of its 56 references that are in the map.
- First posted on arXiv 11 Jul 2023.

## Placed in

- Mechanistic interpretability / [Circuits and mechanistic analysis](https://mlsanity.com/mechinterp/circuits-and-mechanistic-analysis/)

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Progress measures for grokking via mechanistic interpretability](https://mlsanity.com/paper/2301.05217/) (2023)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Scaling Vision Transformers to 22 Billion Parameters](https://mlsanity.com/paper/2302.05442/) (2023)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [Understanding Deep Image Representations by Inverting Them](https://mlsanity.com/paper/1412.0035/) (2014)
- [Understanding Neural Networks Through Deep Visualization](https://mlsanity.com/paper/1506.06579/) (2015)
- [Do Adversarially Robust ImageNet Models Transfer Better?](https://mlsanity.com/paper/2007.08489/) (2020)
- [Understanding the Role of Individual Units in a Deep Neural Network](https://mlsanity.com/paper/2009.05041/) (2020)
- [Partial success in closing the gap between human and machine vision](https://mlsanity.com/paper/2106.07411/) (2021)
- [HIVE: Evaluating the Human Interpretability of Visual Explanations](https://mlsanity.com/paper/2112.03184/) (2021)

## Built on by (most cited first, 18 of 18)

- [Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions](https://mlsanity.com/paper/2310.19775/) (2023)
- [A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions](https://mlsanity.com/paper/2412.05563/) (2024)
- [Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIP](https://mlsanity.com/paper/2308.14179/) (2023)
- [Linear Explanations for Individual Neurons](https://mlsanity.com/paper/2405.06855/) (2024)
- [Identifying Interpretable Visual Features in Artificial and Biological Neural Systems](https://mlsanity.com/paper/2310.11431/) (2023)
- [Understanding Multi-View Transformers](https://mlsanity.com/paper/2510.24907/) (2025)
- [Discovering Influential Neuron Path in Vision Transformers](https://mlsanity.com/paper/2503.09046/) (2025)
- [Choosing the right basis for interpretability: Psychophysical comparison between neuron-based and dictionary-based representations](https://mlsanity.com/paper/2411.03993/) (2024)
- [Evaluating SAE interpretability without explanations](https://mlsanity.com/paper/2507.08473/) (2025)
- [Evaluating Neuron Explanations: A Unified Framework with Sanity Checks](https://mlsanity.com/paper/2506.05774/) (2025)
- [Faithful and Stable Neuron Explanations for Trustworthy Mechanistic Interpretability](https://mlsanity.com/paper/2512.18092/) (2025)
- [AttentionBender: Manipulating Cross-Attention in Video Diffusion Transformers as a Creative Probe](https://mlsanity.com/paper/2604.20936/) (2026)
- [From Feature Visualization to Visual Circuits: Effect of Adversarial Model Manipulation](https://mlsanity.com/paper/2406.01365/) (2024)
- [Beyond Top Activations: Efficient and Reliable Crowdsourced Evaluation of Automated Interpretability](https://mlsanity.com/paper/2506.07985/) (2025)
- [Sparse but not Simpler: A Multi-Level Interpretability Analysis of Vision Transformers](https://mlsanity.com/paper/2603.15919/) (2026)
- [Capability neq Interpretability: Human Interpretability of Vision Foundation Models](https://mlsanity.com/paper/2605.20337/) (2026)
- [From Interpretability Methods to Interpretable Models](https://mlsanity.com/paper/2609.05399/) (2026)
- [Integrating Feature Understandability into Feature Selection for Human-Centred Tabular Explanations](https://mlsanity.com/paper/2610.16627/) (2026)

## Abstract (arXiv)

In light of the recent widespread adoption of AI systems, understanding the internal information processing of neural networks has become increasingly critical. Most recently, machine vision has seen remarkable progress by scaling neural networks to unprecedented levels in dataset and model size. We here ask whether this extraordinary increase in scale also positively impacts the field of mechanistic interpretability. In other words, has our understanding of the inner workings of scaled neural networks improved as well? We use a psychophysical paradigm to quantify one form of mechanistic interpretability for a diverse suite of nine models and find no scaling effect for interpretability - neither for model nor dataset size. Specifically, none of the investigated state-of-the-art models are easier to interpret than the GoogLeNet model from almost a decade ago. Latest-generation vision models appear even less interpretable than older architectures, hinting at a regression rather than improvement, with modern models sacrificing interpretability for accuracy. These results highlight the need for models explicitly designed to be mechanistically interpretable and the need for more helpful interpretability methods to increase our understanding of networks at an atomic level. We release a dataset containing more than 130'000 human responses from our psychophysical evaluation of 767 units across nine models. This dataset facilitates research on automated instead of human-based interpretability evaluations, which can ultimately be leveraged to directly optimize the mechanistic interpretability of models.