mlsanityData to 7 Oct 2026

What Does it Mean for a Neural Network to Learn a "World Model"?

arXiv 2507.21513 ↗First posted 29 Jul 2025cs.AI cs.CLcited by 7

On the map: World models → World models for language agents

Abstract (arXiv)

We propose a set of precise criteria for saying a neural net learns and uses a "world model." The goal is to give an operational meaning to terms that are often used informally, in order to provide a common language for experimental investigation. We focus specifically on the idea of representing a latent "state space" of the world, leaving modeling the effect of actions to future work. Our definition is based on ideas from the linear probing literature, and formalizes the notion of a computation that factors through a representation of the data generation process. An essential addition to the definition is a set of conditions to check that such a "world model" is not a trivial consequence of the neural net's data or task.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 26 of its 47 references that are in the map. First posted on arXiv 29 Jul 2025.

Builds on

26 of 47 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Making Large Language Models into World Models with Precondition and Effect Knowledge18 Sep
  2. 2024Evaluating the World Model Implicit in a Generative Model6 Jun
  3. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Models2 Sep
  4. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought22 Jun

Other references · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  6. 2018World Modelscited by 1,444
  7. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  8. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  9. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  10. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  11. 2024Genie: Generative Interactive Environmentscited by 706
  12. 2019Designing and Interpreting Probes with Control Taskscited by 608
10 more
  1. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
  2. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  3. 2023Language Models Represent Space and Timecited by 306
  4. 2021Implicit Representations of Meaning in Neural Language Modelscited by 179
  5. 2017Learning to Generate Reviews and Discovering Sentimentcited by 147
  6. 2022Language Models as Agent Modelscited by 131
  7. 2021Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Colorcited by 118
  8. 2021Chess as a Testbed for Language Model State Trackingcited by 69
  9. 2023Emergence of Maps in the Memories of Blind Navigation Agentscited by 31
  10. 2023The System Model and the User Model: Exploring AI Dashboard Designcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 6.

7 papers.

  1. 2026Language Models Struggle to Use Representations Learned In-Contextcited by 6
  2. 2025Better World Models Can Lead to Better Post-Training Performancecited by 4
  3. 2026MetaOthello: A Controlled Study of Multiple World Models in Transformerscited by 3
  4. 2026World-Time Compute with Verified Code World Modelscited by 0
  5. 2026World Modeling in Transformerscited by 0
  6. 2026Few-Shot Demonstrations Elicit the Use of In-Context World Representations in LLMscited by 0
  7. 2026Small transformers track Bayesian evidence for latent common causes via a context-invariant mechanismcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Critique of World Model
  2. 2025When Do Neural Networks Learn World Models?
  3. 2026Learning to Theorize the World from Observation
  4. 2026Do World Models Learn Global Understanding?
  5. 2026Latent State Design for World Models under Sufficiency Constraints
  6. 2026Research on World Models Is Not Merely Injecting World Knowledge into Specific Tasks
  7. 2025Improving World Models using Deep Supervision with Linear Probes
  8. 2025Context and Diversity Matter: The Emergence of In-Context Learning in World Models