# From task structures to world models: What do LLMs know?

arXiv: https://arxiv.org/abs/2310.04276 · HTML: https://mlsanity.com/paper/2310.04276/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Placed in World models for language agents, one of 11 directions in World models.
- Builds on 24 of its 114 references that are in the map.
- First posted on arXiv 6 Oct 2023.

## Placed in

- World models / [World models for language agents](https://mlsanity.com/wm/world-models-for-language-agents/)

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Multimodal Few-Shot Learning with Frozen Language Models](https://mlsanity.com/paper/2106.13884/) (2021)
- [Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task](https://mlsanity.com/paper/2210.13382/) (2022)
- [Emergent Linear Representations in World Models of Self-Supervised Sequence Models](https://mlsanity.com/paper/2309.00941/) (2023)
- [Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks](https://mlsanity.com/paper/2307.02477/) (2023)
- [DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning](https://mlsanity.com/paper/2006.08381/) (2020)
- [NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics](https://mlsanity.com/paper/2112.08726/) (2021)
- [Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color](https://mlsanity.com/paper/2109.06129/) (2021)
- [From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought](https://mlsanity.com/paper/2306.12672/) (2023)
- [Tracr: Compiled Transformers as a Laboratory for Interpretability](https://mlsanity.com/paper/2301.05062/) (2023)
- [Neural Machine Translation: A Review and Survey](https://mlsanity.com/paper/1912.02047/) (2019)
- [Meaning without reference in large language models](https://mlsanity.com/paper/2208.02957/) (2022)
- [Eight Things to Know about Large Language Models](https://mlsanity.com/paper/2304.00612/) (2023)
- [People construct simplified mental representations to plan](https://mlsanity.com/paper/2105.06948/) (2021)
- [Emergent Representations of Program Semantics in Language Models Trained on Programs](https://mlsanity.com/paper/2305.11169/) (2023)
- [TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI](https://mlsanity.com/paper/2306.06924/) (2023)
- [Modeling human intuitions about liquid flow with particle-based simulation](https://mlsanity.com/paper/1809.01524/) (2018)
- [Physical problem solving: Joint planning with symbolic, geometric, and dynamic constraints](https://mlsanity.com/paper/1707.08212/) (2017)

## Built on by (most cited first, 30 of 46)

- [Elements of World Knowledge (EWoK): A Cognition-Inspired Framework for Evaluating Basic World Knowledge in Language Models](https://mlsanity.com/paper/2405.09605/) (2024)
- [Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMs](https://mlsanity.com/paper/2311.09730/) (2023)
- [Actions Speak Louder than Words: Agent Decisions Reveal Implicit Biases in Language Models](https://mlsanity.com/paper/2501.17420/) (2025)
- [Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2504.14891/) (2025)
- [SHED: Shapley-Based Automated Dataset Refinement for Instruction Fine-Tuning](https://mlsanity.com/paper/2405.00705/) (2024)
- [On Large Language Models in National Security Applications](https://mlsanity.com/paper/2407.03453/) (2024)
- [A Philosophical Introduction to Language Models - Part II: The Way Forward](https://mlsanity.com/paper/2405.03207/) (2024)
- [Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Review](https://mlsanity.com/paper/2401.01519/) (2024)
- [A Survey on Large Language Models for Automated Planning](https://mlsanity.com/paper/2502.12435/) (2025)
- [L-MTP: Leap Multi-Token Prediction Beyond Adjacent Context for Large Language Models](https://mlsanity.com/paper/2505.17505/) (2025)
- [Does ChatGPT Have a Mind?](https://mlsanity.com/paper/2407.11015/) (2024)
- [Are Language Models More Like Libraries or Like Librarians? Bibliotechnism, the Novel Reference Problem, and the Attitudes of LLMs](https://mlsanity.com/paper/2401.04854/) (2024)
- [The Imperative of Conversation Analysis in the Era of LLMs: A Survey of Tasks, Techniques, and Trends](https://mlsanity.com/paper/2409.14195/) (2024)
- [Vision Language Models See What You Want but not What You See](https://mlsanity.com/paper/2410.00324/) (2024)
- [Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Framework](https://mlsanity.com/paper/2506.15538/) (2025)
- [Evolving Roles of LLMs in Scientific Innovation: Assistant, Collaborator, Scientist, and Evaluator](https://mlsanity.com/paper/2507.11810/) (2025)
- [When Do Neural Networks Learn World Models?](https://mlsanity.com/paper/2502.09297/) (2025)
- [Transforming Agency. On the mode of existence of Large Language Models](https://mlsanity.com/paper/2407.10735/) (2024)
- [Reflection-Bench: Evaluating Epistemic Agency in Large Language Models](https://mlsanity.com/paper/2410.16270/) (2024)
- [What Do Large Language Models Know? Tacit Knowledge as a Potential Causal-Explanatory Structure](https://mlsanity.com/paper/2504.12187/) (2025)
- [Linear Spatial World Models Emerge in Large Language Models](https://mlsanity.com/paper/2506.02996/) (2025)
- [CogDriver: Integrating Cognitive Inertia for Temporally Coherent Planning in Autonomous Driving](https://mlsanity.com/paper/2509.00789/) (2025)
- [Can structural correspondences ground real world representational content in Large Language Models?](https://mlsanity.com/paper/2506.16370/) (2025)
- [SC2Arena and StarEvolve: Benchmark and Self-Improvement Framework for LLMs in Complex Decision-Making Tasks](https://mlsanity.com/paper/2508.10428/) (2025)
- [Chatting with Bots: AI, Speech Acts, and the Edge of Assertion](https://mlsanity.com/paper/2410.16645/) (2024)
- [Revisiting Rogers' Paradox in the Context of Human-AI Interaction](https://mlsanity.com/paper/2501.10476/) (2025)
- [Shades of Zero: Distinguishing Impossibility from Inconceivability](https://mlsanity.com/paper/2502.20469/) (2025)
- [Propose and Rectify: A Forensics-Driven MLLM Framework for Image Manipulation Localization](https://mlsanity.com/paper/2508.17976/) (2025)
- [From Next Token Prediction to (STRIPS) World Models](https://mlsanity.com/paper/2509.13389/) (2025)
- [Do LLMs Build Spatial World Models? Evidence from Grid-World Maze Tasks](https://mlsanity.com/paper/2604.10690/) (2026)

## Abstract (arXiv)

In what sense does a large language model have knowledge? The answer to this question extends beyond the capabilities of a particular AI system, and challenges our assumptions about the nature of knowledge and intelligence. We answer by granting LLMs "instrumental knowledge"; knowledge defined by a certain set of abilities. We then ask how such knowledge is related to the more ordinary, "worldly" knowledge exhibited by human agents, and explore this in terms of the degree to which instrumental knowledge can be said to incorporate the structured world models of cognitive science. We discuss ways LLMs could recover degrees of worldly knowledge, and suggest such recovery will be governed by an implicit, resource-rational tradeoff between world models and task demands.