A Model of Understanding in Deep Learning Systems
Not on the map. Why?
Abstract (arXiv)
I propose a model of systematic understanding, suitable for machine learning systems. On this account, an agent understands a property of a target system when it contains an adequate internal model that tracks real regularities, is coupled to the target by stable bridge principles, and supports reliable prediction. I argue that contemporary deep learning systems often can and do achieve such understanding. However they generally fall short of the ideal of scientific understanding: the understanding is symbolically misaligned with the target system, not explicitly reductive, and only weakly unifying. I label this the Fractured Understanding Hypothesis.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 34 of its 185 references that are in the map. First posted on arXiv 5 Apr 2026.
Builds on
34 of 185 references.
References · 34
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2023GPT-4 Technical Reportcited by 18,396
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
- 2022Emergent Abilities of Large Language Modelscited by 2,512
- 2016Understanding deep learning requires rethinking generalizationcited by 1,674
- 2016Concrete Problems in AI Safetycited by 1,642
22 more
- 2018World Modelscited by 1,513
- 2016Understanding intermediate layers using linear classifier probescited by 1,353
- 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
- 2022Toy Models of Superpositioncited by 971
- 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 629
- 2017Deep Learning Scaling is Predictable, Empiricallycited by 582
- 2020Improving robustness against common corruptions by covariate shift adaptationcited by 431
- 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
- 2020Lagrangian Neural Networkscited by 398
- 2018Deep Learning: A Critical Appraisalcited by 217
- 2018Deep learning generalizes because the parameter-function map is biased towards simple functionscited by 183
- 2022The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenoncited by 94
- 2021Learning in High Dimension Always Amounts to Extrapolationcited by 66
- 2023Rigorously Assessing Natural Language Explanations of Neuronscited by 45
- 2019Where is the Information in a Deep Neural Network?cited by 42
- 2024A Philosophical Introduction to Language Models -- Part I: Continuity With Classic Debatescited by 23
- 2025Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesiscited by 22
- 2024A Philosophical Introduction to Language Models - Part II: The Way Forwardcited by 18
- 2024Memorization in deep learning: A surveycited by 15
- 2022The Curious Case of Benign Memorizationcited by 12
- 2025Mechanistic Indicators of Understanding in Large Language Modelscited by 8
- 2025Deflating Deflationism: A Critical Perspective on Debunking Arguments Against LLM Mentalitycited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.