mlsanityData to 7 Oct 2026

Levels of Analysis for Large Language Models

arXiv 2503.13401 ↗First posted 17 Mar 2025cs.CL cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

Modern artificial intelligence systems, such as large language models, are increasingly powerful but also increasingly hard to understand. Recognizing this problem as analogous to the historical difficulties in understanding the human mind, we argue that methods developed in cognitive science can be useful for understanding large language models. We propose a framework for applying these methods based on the levels of analysis that David Marr proposed for studying information processing systems. By revisiting established cognitive science techniques relevant to each level and illustrating their potential to yield insights into the behavior and internal organization of large language models, we aim to provide a toolkit for making sense of these new kinds of minds.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 57 of its 142 references that are in the map. First posted on arXiv 17 Mar 2025.

Builds on

57 of 142 references.

References · 57

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  9. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  10. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  11. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  12. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
38 more
  1. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  2. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  3. 2022In-context Learning and Induction Headscited by 930
  4. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  5. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  6. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  7. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 533
  8. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
  9. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  10. 2022Using cognitive psychology to understand GPT-3cited by 389
  11. 2023Language Models Represent Space and Timecited by 306
  12. 2022Emergent Analogical Reasoning in Large Language Modelscited by 239
  13. 2022Towards Understanding Grokking: An Effective Theory of Representation Learningcited by 229
  14. 2024An Introduction to Vision-Language Modelingcited by 175
  15. 2023Getting aligned on representational alignmentcited by 142
  16. 2024Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worsecited by 107
  17. 2023Why think step by step? Reasoning emerges from the locality of experiencecited by 103
  18. 2024ARC Prize 2024: Technical Reportcited by 103
  19. 2024Understanding the Limits of Vision Language Models Through the Lens of the Binding Problemcited by 71
  20. 2023The Foundation Model Transparency Indexcited by 63
  21. 2024CogBench: a large language model walks into a psychology labcited by 60
  22. 2024Emergent World Models and Latent Variable Estimation in Chess-Playing Language Modelscited by 60
  23. 2024One fish, two fish, but not the whole sea: Alignment reduces language models' conceptual diversitycited by 57
  24. 2024Auxiliary task demands mask the capabilities of smaller language modelscited by 54
  25. 2024Measuring Implicit Bias in Explicitly Unbiased Large Language Modelscited by 49
  26. 2024Large Language Models Assume People are More Rational than We Really arecited by 49
  27. 2023Large language models predict human sensory judgments across six modalitiescited by 47
  28. 2022Can language models handle recursively nested grammatical structures? A case study on comparing models and humanscited by 41
  29. 2024Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMscited by 39
  30. 2020Understanding Human Intelligence through Human Limitationscited by 33
  31. 2024Incoherent Probability Judgments in Large Language Modelscited by 29
  32. 2023Bayes in the age of intelligent machinescited by 22
  33. 2024Rational Metareasoning for Large Language Modelscited by 20
  34. 2024Eliciting the Priors of Large Language Models using Iterated In-Context Learningcited by 19
  35. 2019Towards a Quantum-Like Cognitive Architecture for Decision-Makingcited by 17
  36. 2023Gromov-Wasserstein unsupervised alignment reveals structural correspondences between the color similarity structures of humans and large language modelscited by 16
  37. 2023Conceptual structure coheres in human cognition but not in large language modelscited by 14
  38. 2025What is a Number, That a Large Language Model May Know It?cited by 14

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 6.

9 papers.

  1. 2025Are Large Language Models Reliable AI Scientists? Assessing Reverse-Engineering of Black-Box Systemscited by 13
  2. 2025The Other Mind: How Language Models Exhibit Human Temporal Cognitioncited by 4
  3. 2025Implicit In-Context Learning: Evidence from Artificial Language Experimentscited by 2
  4. 2026Reasoning aligns language models to human cognitioncited by 2
  5. 2026The Case for Model Science: Verify, Explore, Steer, Refinecited by 2
  6. 2026Investigating Concept Alignment Using Implausible Category Memberscited by 0
  7. 2026Metacognition Should Be the Scientific Framework for Bounded and Effective Self-Governance in Generative AIcited by 0
  8. 2026Process-Constituted Intelligence: A Shared Criterion for Humans and Machinescited by 0
  9. 2026Assessing mentalization in humans and large language modelscited by 0