mlsanityData to 14 Oct 2026

The Debate Over Understanding in AI's Large Language Models

arXiv 2210.13966 ↗First posted 14 Oct 2022cs.LG cs.AIcited by 142

Not on the map. Why?

Abstract (arXiv)

We survey a current, heated debate in the AI research community on whether large pre-trained language models can be said to "understand" language -- and the physical and social situations language encodes -- in any important sense. We describe arguments that have been made for and against such understanding, and key questions for the broader sciences of intelligence that have arisen in light of these arguments. We contend that a new science of intelligence can be developed that will provide insight into distinct modes of understanding, their strengths and limitations, and the challenge of integrating diverse forms of cognition.

Key facts. Cited by 142 papers in the map (data through 14 Oct 2026). Builds on 33 of its 97 references that are in the map. First posted on arXiv 14 Oct 2022.

Builds on

33 of 97 references.

References · 33

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  7. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  8. 2022Emergent Abilities of Large Language Modelscited by 2,512
  9. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  10. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  11. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  12. 2022In-context Learning and Induction Headscited by 964
21 more
  1. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 876
  2. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  3. 2018Annotation Artifacts in Natural Language Inference Datacited by 671
  4. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 463
  5. 2022Using cognitive psychology to understand GPT-3cited by 398
  6. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 365
  7. 2019Probing Neural Network Comprehension of Natural Language Argumentscited by 214
  8. 2021Implicit Representations of Meaning in Neural Language Modelscited by 180
  9. 2022Language models show human-like content effects on reasoning taskscited by 173
  10. 2020How Can We Accelerate Progress Towards Human-like Linguistic Generalization?cited by 114
  11. 2021Competency Problems: On Finding and Removing Artifacts in Language Datacited by 91
  12. 2022Meaning without reference in large language modelscited by 76
  13. 2018Theoretical Impediments to Machine Learning With Seven Sparks from the Causal Revolutioncited by 66
  14. 2022Do Large Language Models know what humans know?cited by 60
  15. 2017The Argument Reasoning Comprehension Task: Identification and Reconstruction of Implicit Warrantscited by 56
  16. 2020Word meaning in minds and machinescited by 51
  17. 2022Neurocompositional computing: From the Central Paradox of Cognition to a new generation of AI systemscited by 44
  18. 2022Large Language Models and the Reverse Turing Testcited by 38
  19. 2022What Do NLP Researchers Believe? Results of the NLP Community Metasurveycited by 28
  20. 2022Machine Reading, Fast and Slow: When Do Models "Understand" Language?cited by 13
  21. 2022Predicting Human Psychometric Properties Using Computational Language Modelscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 39; 2024: 30; 2025: 38; 2026: 33.

142 papers.

Sort:

By area:

  1. 2023Personality Traits in Large Language Modelscited by 169
  2. 2023The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domaincited by 101
  3. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  4. 2024Pandora: Towards General World Model with Natural Language Actions and Video Statescited by 86
  5. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Surveycited by 83
  6. 2023Turning large language models into cognitive modelscited by 67
  7. 2023Boosting Theory-of-Mind Performance in Large Language Models via Promptingcited by 64
  8. 2024LLMs achieve adult human performance on higher-order theory of mind taskscited by 61
  9. 2023Do LLM Agents Exhibit Social Behavior?cited by 53
  10. 2025How Linguistics Learned to Stop Worrying and Love the Language Modelscited by 51
  11. 2024Large Language Models and Cognitive Science: A Comprehensive Review of Similarities, Differences, and Challengescited by 49
  12. 2024People cannot distinguish GPT-4 from a human in a Turing testcited by 48
38 more
  1. 2025Large Language Models Pass the Turing Testcited by 48
  2. 2023Testing AI on language comprehension tasks reveals insensitivity to underlying meaningcited by 45
  3. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approachescited by 43
  4. 2024Core Knowledge Deficits in Multi-Modal Language Modelscited by 38
  5. 2025The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMscited by 36
  6. 2024Benchmarks as Microscopes: A Call for Model Metrologycited by 35
  7. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Surveycited by 31
  8. 2023Emergent Representations of Program Semantics in Language Models Trained on Programscited by 30
  9. 2023Do Vision and Language Models Share Concepts? A Vector Space Alignment Studycited by 27
  10. 2023Creating a Large Language Model of a Philosophercited by 21
  11. 2023The feasibility of artificial consciousness through the lens of neurosciencecited by 18
  12. 2025Context parroting: A simple but tough-to-beat baseline for foundation models in scientific machine learningcited by 18
  13. 2023MEWL: Few-shot multimodal word learning with referential uncertaintycited by 17
  14. 2023A Sentence is Worth a Thousand Pictures: Can Large Language Models Understand Hum4n L4ngu4ge and the W0rld behind W0rds?cited by 17
  15. 2024Large Language Model Benchmarks in Medical Taskscited by 17
  16. 2023Separating form and meaning: Using self-consistency to quantify task understanding across multiple sensescited by 15
  17. 2023Studying and improving reasoning in humans and machinescited by 14
  18. 2023Evaluating the Deductive Competence of Large Language Modelscited by 12
  19. 2025Credible Plan-Driven RAG Method for Multi-Hop Question Answeringcited by 12
  20. 2023CancerGPT: Few-shot Drug Pair Synergy Prediction using Large Pre-trained Language Modelscited by 11
  21. 2024Are Language Models More Like Libraries or Like Librarians? Bibliotechnism, the Novel Reference Problem, and the Attitudes of LLMscited by 11
  22. 2024Anchor function: a type of benchmark functions for studying language modelscited by 11
  23. 2024Is artificial consciousness achievable? Lessons from the human braincited by 11
  24. 2024Position: An Inner Interpretability Framework for AI Inspired by Lessons from Cognitive Neurosciencecited by 11
  25. 2024Collective Innovation in Groups of Large Language Modelscited by 11
  26. 2024Delving into the Reversal Curse: How Far Can Large Language Models Generalize?cited by 11
  27. 2025The Stochastic Parrot on LLM's Shoulder: A Summative Assessment of Physical Concept Understandingcited by 11
  28. 2025Epistemological Fault Lines Between Human and Artificial Intelligencecited by 11
  29. 2023Implicit meta-learning may lead language models to trust more reliable sourcescited by 10
  30. 2024Vision Language Models See What You Want but not What You Seecited by 10
  31. 2023Comparing Machines and Children: Using Developmental Psychology Experiments to Assess the Strengths and Weaknesses of LaMDA Responsescited by 9
  32. 2022Deanthropomorphising NLP: Can a Language Model Be Conscious?cited by 8
  33. 2024From Text to Life: On the Reciprocal Relationship between Artificial Life and Large Language Modelscited by 8
  34. 2025Mechanistic Indicators of Understanding in Large Language Modelscited by 8
  35. 2025Readers Prefer Outputs of AI Trained on Copyrighted Books over Expert Human Writerscited by 8
  36. 2023On pitfalls (and advantages) of sophisticated large language modelscited by 7
  37. 2023Hybrid Approaches for Moral Value Alignment in AI Agents: a Manifestocited by 7
  38. 2024Dissecting the Ullman Variations with a SCALPEL: Why do LLMs fail at Trivial Alterations to the False Belief Task?cited by 7

50 most cited of 142 All 142 (JSON)