mlsanityData to 7 Oct 2026

From task structures to world models: What do LLMs know?

arXiv 2310.04276 ↗First posted 6 Oct 2023cs.AI q-bio.NCcited by 46

On the map: World models → World models for language agents

Abstract (arXiv)

In what sense does a large language model have knowledge? The answer to this question extends beyond the capabilities of a particular AI system, and challenges our assumptions about the nature of knowledge and intelligence. We answer by granting LLMs "instrumental knowledge"; knowledge defined by a certain set of abilities. We then ask how such knowledge is related to the more ordinary, "worldly" knowledge exhibited by human agents, and explore this in terms of the degree to which instrumental knowledge can be said to incorporate the structured world models of cognitive science. We discuss ways LLMs could recover degrees of worldly knowledge, and suggest such recovery will be governed by an implicit, resource-rational tradeoff between world models and task demands.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 24 of its 114 references that are in the map. First posted on arXiv 6 Oct 2023.

Builds on

24 of 114 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Models2 Sep
  2. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought22 Jun
  3. 2023Emergent Representations of Program Semantics in Language Models Trained on Programs18 May
  4. 2020Language Models are Few-Shot Learners28 May

Other references · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  7. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  8. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  9. 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Taskscited by 243
  10. 2020DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learningcited by 159
  11. 2021NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristicscited by 153
  12. 2021Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Colorcited by 118
8 more
  1. 2019Neural Machine Translation: A Review and Surveycited by 99
  2. 2023Tracr: Compiled Transformers as a Laboratory for Interpretabilitycited by 99
  3. 2022Meaning without reference in large language modelscited by 76
  4. 2023Eight Things to Know about Large Language Modelscited by 73
  5. 2021People construct simplified mental representations to plancited by 45
  6. 2023TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AIcited by 25
  7. 2018Modeling human intuitions about liquid flow with particle-based simulationcited by 12
  8. 2017Physical problem solving: Joint planning with symbolic, geometric, and dynamic constraintscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 13; 2025: 22; 2026: 10.

46 papers.

Sort:

By area:

  1. 2024Elements of World Knowledge (EWoK): A Cognition-Inspired Framework for Evaluating Basic World Knowledge in Language Modelscited by 55
  2. 2023Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMscited by 36
  3. 2025Actions Speak Louder than Words: Agent Decisions Reveal Implicit Biases in Language Modelscited by 30
  4. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Surveycited by 20
  5. 2024SHED: Shapley-Based Automated Dataset Refinement for Instruction Fine-Tuningcited by 19
  6. 2024On Large Language Models in National Security Applicationscited by 18
  7. 2024Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Reviewcited by 17
  8. 2024A Philosophical Introduction to Language Models - Part II: The Way Forwardcited by 17
  9. 2025A Survey on Large Language Models for Automated Planningcited by 17
  10. 2025L-MTP: Leap Multi-Token Prediction Beyond Adjacent Context for Large Language Modelscited by 14
  11. 2024Does ChatGPT Have a Mind?cited by 13
  12. 2024Are Language Models More Like Libraries or Like Librarians? Bibliotechnism, the Novel Reference Problem, and the Attitudes of LLMscited by 11
34 more
  1. 2024The Imperative of Conversation Analysis in the Era of LLMs: A Survey of Tasks, Techniques, and Trendscited by 11
  2. 2024Vision Language Models See What You Want but not What You Seecited by 10
  3. 2025Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Frameworkcited by 9
  4. 2025Evolving Roles of LLMs in Scientific Innovation: Assistant, Collaborator, Scientist, and Evaluatorcited by 9
  5. 2025When Do Neural Networks Learn World Models?cited by 8
  6. 2024Transforming Agency. On the mode of existence of Large Language Modelscited by 7
  7. 2024Reflection-Bench: Evaluating Epistemic Agency in Large Language Modelscited by 7
  8. 2025What Do Large Language Models Know? Tacit Knowledge as a Potential Causal-Explanatory Structurecited by 5
  9. 2025CogDriver: Integrating Cognitive Inertia for Temporally Coherent Planning in Autonomous Drivingcited by 5
  10. 2025Linear Spatial World Models Emerge in Large Language Modelscited by 4
  11. 2025Can structural correspondences ground real world representational content in Large Language Models?cited by 3
  12. 2025SC2Arena and StarEvolve: Benchmark and Self-Improvement Framework for LLMs in Complex Decision-Making Taskscited by 3
  13. 2024Chatting with Bots: AI, Speech Acts, and the Edge of Assertioncited by 2
  14. 2025Revisiting Rogers' Paradox in the Context of Human-AI Interactioncited by 2
  15. 2025Shades of Zero: Distinguishing Impossibility from Inconceivabilitycited by 2
  16. 2025Propose and Rectify: A Forensics-Driven MLLM Framework for Image Manipulation Localizationcited by 2
  17. 2025From Next Token Prediction to (STRIPS) World Modelscited by 2
  18. 2026Do LLMs Build Spatial World Models? Evidence from Grid-World Maze Taskscited by 2
  19. 2024A Survey for Deep Reinforcement Learning Based Network Intrusion Detectioncited by 1
  20. 2025Contemplative Artificial Intelligencecited by 1
  21. 2025Targeted control of fast prototyping through domain-specific interfacecited by 1
  22. 2026A Checklist for Trustworthy, Safe, and User-Friendly Mental Health Chatbotscited by 1
  23. 2026No More, No Less: Least-Privilege Language Modelscited by 1
  24. 2025Representation in large language modelscited by 0
  25. 2025Deflating Deflationism: A Critical Perspective on Debunking Arguments Against LLM Mentalitycited by 0
  26. 2025What do language models model? Transformers, automata, and the format of thoughtcited by 0
  27. 2025Enhancing Computational Cognitive Architectures with LLMs: A Case Studycited by 0
  28. 2026STO-RL: Offline RL under Sparse Rewards via LLM-Guided Subgoal Temporal Ordercited by 0
  29. 2026Extending Minimal Pairs with Ordinal Surprisal Curves and Entropy Across Applied Domainscited by 0
  30. 2026How LLMs Might Thinkcited by 0
  31. 2026Geo-Spatial Concept Probing of Large Language Models: Abstraction, Compositionality, and Groundingcited by 0
  32. 2026Map Users and Mapmakers: The Scope of Cognitive Attribution from Acquired Representationscited by 0
  33. 2026Few-Shot Demonstrations Elicit the Use of In-Context World Representations in LLMscited by 0
  34. 2026Anthropomorphism in the age of Large Language Models: An overview of potential risks and mitigationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Human Cognition in Machines: A Unified Perspective of World Models
  2. 2023Brain in a Vat: On Missing Pieces Towards Artificial General Intelligence in Large Language Models
  3. 2025Recognizing Artificial Minds: A Philosophical Defense of AI Cognition
  4. 2025Beyond World Models: Rethinking Understanding in AI Models
  5. 2026LLM and Human Modes of Representation
  6. 2026Understanding Large Language Models
  7. 2025What Do LLM Agents Do When Left Alone? Evidence of Spontaneous Meta-Cognitive Patterns
  8. 2026What Do LLM Agents Know About Their World? Task2Quiz: A Paradigm for Studying Environment Understanding