mlsanityData to 14 Oct 2026

When Do Neural Networks Learn World Models?

arXiv 2502.09297 ↗First posted 13 Feb 2025cs.LGcited by 8

On the map: World models → World models for language agents

Abstract (arXiv)

Humans develop world models that capture the underlying generation process of data. Whether neural networks can learn similar world models remains an open problem. In this work, we present the first theoretical results for this problem, showing that in a multi-task setting, models with a low-degree bias provably recover latent data-generating variables under mild assumptions--even if proxy tasks involve complex, non-linear functions of the latents. However, such recovery is sensitive to model architecture. Our analysis leverages Boolean models of task solutions via the Fourier-Walsh transform and introduces new techniques for analyzing invertible Boolean transforms, which may be of independent interest. We illustrate the algorithmic implications of our results and connect them to related research areas, including self-supervised learning, out-of-distribution generalization, and the linear representation hypothesis in large language models.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 68 of its 96 references that are in the map. First posted on arXiv 13 Feb 2025.

Builds on

68 of 96 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Making Large Language Models into World Models with Precondition and Effect Knowledge18 Sep
  2. 2024Robust agents learn causal world models16 Feb
  3. 2023From task structures to world models: What do LLMs know?6 Oct
  4. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Models2 Sep
  5. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought22 Jun
  6. 2023Emergent Representations of Program Semantics in Language Models Trained on Programs18 May
  7. 2020Language Models are Few-Shot Learners28 May

Other references · 61

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  5. 2019Invariant Risk Minimizationcited by 1,854
  6. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  7. 2018World Modelscited by 1,513
  8. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
  9. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  10. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  11. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  12. 2020Self-supervised Learning: Generative or Contrastivecited by 694
38 more
  1. 2017Spectrally-normalized margin bounds for neural networkscited by 686
  2. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 675
  3. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 655
  4. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 550
  5. 2019Benign Overfitting in Linear Regressioncited by 549
  6. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  7. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 503
  8. 2021A White Paper on Neural Network Quantizationcited by 472
  9. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
  10. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 407
  11. 2024The Platonic Representation Hypothesiscited by 382
  12. 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 368
  13. 2023Language Models Represent Space and Timecited by 316
  14. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 312
  15. 2020Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learningcited by 297
  16. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 282
  17. 2020Inductive Biases for Deep Learning of Higher-Level Cognitioncited by 275
  18. 2020Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Losscited by 268
  19. 2018Characterizing Implicit Bias in Terms of Optimization Geometrycited by 266
  20. 2018Implicit Bias of Gradient Descent on Linear Convolutional Networkscited by 251
  21. 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Taskscited by 246
  22. 2020How Neural Networks Extrapolate: From Feedforward to Graph Neural Networkscited by 222
  23. 2023An Overview of Catastrophic AI Riskscited by 222
  24. 2021Contrastive Learning Inverts the Data Generating Processcited by 221
  25. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 212
  26. 2022Exploring Length Generalization in Large Language Modelscited by 206
  27. 2019SGD on Neural Networks Learns Functions of Increasing Complexitycited by 204
  28. 2018Deep learning generalizes because the parameter-function map is biased towards simple functionscited by 183
  29. 2021Implicit Representations of Meaning in Neural Language Modelscited by 180
  30. 2019What Can Neural Networks Reason About?cited by 159
  31. 2021The Low-Rank Simplicity Bias in Deep Networkscited by 154
  32. 2020Predicting What You Already Know Helps: Provable Self-Supervised Learningcited by 152
  33. 2020Contrastive learning, multi-view redundancy, and linear modelscited by 146
  34. 2022Interventional Causal Representation Learningcited by 142
  35. 2018Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approachcited by 126
  36. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 106
  37. 2020Neural Networks Fail to Learn Periodic Functions and How to Fix Itcited by 100
  38. 2021Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Biascited by 83

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 5.

8 papers.

  1. 2025I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?cited by 13
  2. 2025Exploring the Hidden Reasoning Process of Large Language Models by Misleading Themcited by 3
  3. 2025Prediction horizon shapes representations in predictive learningcited by 0
  4. 2026Reconciling In-Context and In-Weight Learning via Dual Representation Space Encodingcited by 0
  5. 2026From Generalist to Specialist Representationcited by 0
  6. 2026Quantifying and Optimizing Simplicity via Polynomial Representationscited by 0
  7. 2026The FIL Hypothesis: Inductive Biases Help with Kernel Engineeringcited by 0
  8. 2026Few-Shot Demonstrations Elicit the Use of In-Context World Representations in LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025What Does it Mean for a Neural Network to Learn a "World Model"?
  2. 2026Discrete World Models via Regularization
  3. 2025Better World Models Can Lead to Better Post-Training Performance
  4. 2026Task Ecologies and the Evolution of World-Tracking Representations in Large Language Models
  5. 2024Transformers Use Causal World Models in Maze-Solving Tasks
  6. 2024Disentangling Representations through Multi-task Learning
  7. 2025Information Structure in Mappings: An Approach to Learning, Representation, and Generalisation
  8. 2025Characterising the Inductive Biases of Neural Networks on Boolean Data