mlsanityData to 7 Oct 2026

Dissociating language and thought in large language models

arXiv 2301.06627 ↗First posted 16 Jan 2023cs.CL cs.AIcited by 27

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large Language Models (LLMs) have come closest among all models to date to mastering human language, yet opinions about their linguistic and cognitive capabilities remain split. Here, we evaluate LLMs using a distinction between formal linguistic competence -- knowledge of linguistic rules and patterns -- and functional linguistic competence -- understanding and using language in the world. We ground this distinction in human neuroscience, which has shown that formal and functional competence rely on different neural mechanisms. Although LLMs are surprisingly good at formal competence, their performance on functional competence tasks remains spotty and often requires specialized fine-tuning and/or coupling with external modules. We posit that models that use language in human-like ways would need to master both of these competence types, which, in turn, could require the emergence of mechanisms specialized for formal linguistic competence, distinct from functional competence.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 127 of its 424 references that are in the map. First posted on arXiv 16 Jan 2023.

Builds on

127 of 424 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Measuring and Narrowing the Compositionality Gap in Language Models7 Oct
  2. 2022On the Paradox of Learning to Reason from Data23 May
  3. 2022Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning tasks11 May
  4. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Models30 Nov

Other references · 123

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  9. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  10. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  11. 2017Deep reinforcement learning from human preferencescited by 3,846
  12. 2020Longformer: The Long-Document Transformercited by 3,362
38 more
  1. 2022Emergent Abilities of Large Language Modelscited by 2,483
  2. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  3. 2019Language Models as Knowledge Bases?cited by 1,922
  4. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  5. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  6. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  7. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  8. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  9. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  10. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  11. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  12. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  13. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  14. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  15. 2019How multilingual is Multilingual BERT?cited by 642
  16. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  17. 2019Designing and Interpreting Probes with Control Taskscited by 608
  18. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  19. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 553
  20. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 428
  21. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  22. 2019The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervisioncited by 369
  23. 2019CamemBERT: a Tasty French Language Modelcited by 368
  24. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 361
  25. 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 350
  26. 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
  27. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
  28. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
  29. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 304
  30. 2018Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understandingcited by 304
  31. 2019Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoningcited by 281
  32. 2018Analysis Methods in Neural Language Processing: A Surveycited by 276
  33. 2019Measuring Bias in Contextualized Word Representationscited by 275
  34. 2019Compositionality decomposed: how do neural networks generalise?cited by 264
  35. 2022Can language models learn from explanations in context?cited by 248
  36. 2019Analyzing the Structure of Attention in a Transformer Language Modelcited by 241
  37. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 240
  38. 2018Targeted Syntactic Evaluation of Language Modelscited by 239

50 most cited of 123 All 123 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 8; 2025: 1; 2026: 10.

27 papers.

Sort:

By area:

  1. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  2. 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 175
  3. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thoughtcited by 118
  4. 2025Agentic Large Language Models, a surveycited by 105
  5. 2024A Vision Check-up for Language Modelscited by 36
  6. 2024I Think, Therefore I am: Benchmarking Awareness of Large Language Models Using AwareBenchcited by 20
  7. 2024Language in Vivo vs. in Silico: Size Matters but Larger Language Models Still Do Not Comprehend Language on a Par with Humans Due to Impenetrable Semantic Referencecited by 15
  8. 2024A Systematic Analysis of Large Language Models as Soft Reasoners: The Case of Syllogistic Inferencescited by 13
  9. 2023Knowledge Engineering using Large Language Modelscited by 9
  10. 2023Are NLP Models Good at Tracing Thoughts: An Overview of Narrative Understandingcited by 8
  11. 2023Learning a Structural Causal Model for Intuition Reasoning in Conversationcited by 7
  12. 2024Sub-goal Distillation: A Method to Improve Small Language Agentscited by 6
15 more
  1. 2023Frugal LMs Trained to Invoke Symbolic Solvers Achieve Parameter-Efficient Arithmetic Reasoningcited by 4
  2. 2024Stars, Stripes, and Silicon: Unravelling the ChatGPT's All-American, Monochrome, Cis-centric Biascited by 4
  3. 2024CLEVR-POC: Reasoning-Intensive Visual Question Answering in Partially Observable Environmentscited by 3
  4. 2024Can formal argumentative reasoning enhance LLMs performances?cited by 3
  5. 2026CRAFT: Grounded Multi-Agent Coordination Under Partial Informationcited by 3
  6. 2026Sparse Autoencoders Map Brain-LLM Alignment onto Cortical Semantic Topographycited by 2
  7. 2023Attention, not scale, drives human-AI alignment in multimodal language predictioncited by 1
  8. 2026Do Language Models Know What Not to Say? Causal Evidence for Statistical Preemption in LLMscited by 1
  9. 2026Model Collapse as Cultural Evolutioncited by 1
  10. 2026World-Model Collapse as a Phase Transitioncited by 1
  11. 2026The Presupposition Problem in Representation Genesiscited by 0
  12. 2026Heterogeneity in Formal Linguistic Competence of Language Models: Is Data the Real Bottleneck?cited by 0
  13. 2026A Lightweight Multi-Agent Framework for Automated Concrete Barrier Designcited by 0
  14. 2026Thinking effort aligns between humans and reasoning models in abductive reasoningcited by 0
  15. 2026Cognitive Expert Language Models Better Align with the Corresponding Brain Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024The LLM Language Network: A Neuroscientific Approach for Identifying Causally Task-Relevant Units
  2. 2026Understanding Large Language Models
  3. 2025The Emergence of Abstract Thought in Large Language Models Beyond Any Language
  4. 2025Do Large Language Models Think Like the Brain? Sentence-Level Evidences from Layer-Wise Embeddings and fMRI
  5. 2024A Philosophical Introduction to Language Models -- Part I: Continuity With Classic Debates
  6. 2024Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Models
  7. 2026When Language Models Lose Their Mind: The Consequences of Brain Misalignment
  8. 2023Instruction-tuning Aligns LLMs to the Human Brain