mlsanityData to 7 Oct 2026

What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models

arXiv 2507.06952 ↗First posted 9 Jul 2025cs.LG cs.AIcited by 46

On the map: World models → World models for language agents

Abstract (arXiv)

Foundation models are premised on the idea that sequence prediction can uncover deeper domain understanding, much like how Kepler's predictions of planetary motion later led to the discovery of Newtonian mechanics. However, evaluating whether these models truly capture deeper structure remains a challenge. We develop a technique for evaluating foundation models that examines how they adapt to synthetic datasets generated from some postulated world model. Our technique measures whether the foundation model's inductive bias aligns with the world model, and so we refer to it as an inductive bias probe. Across multiple domains, we find that foundation models can excel at their training tasks yet fail to develop inductive biases towards the underlying world model when adapted to new tasks. We particularly find that foundation models trained on orbital trajectories consistently fail to apply Newtonian mechanics when adapted to new physics tasks. Further analysis reveals that these models behave as if they develop task-specific heuristics that fail to generalize.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 39 of its 69 references that are in the map. First posted on arXiv 9 Jul 2025.

Builds on

39 of 69 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Transformers Use Causal World Models in Maze-Solving Tasks16 Dec
  2. 2024Evaluating the World Model Implicit in a Generative Model6 Jun
  3. 2024Emergent World Models and Latent Variable Estimation in Chess-Playing Language Models21 Mar
  4. 2023Linear Latent World Models in Simple Transformers: A Case Study on Othello-GPT11 Oct
  5. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Models2 Sep

Other references · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  4. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
  5. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  6. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  7. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  8. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  9. 2019Designing and Interpreting Probes with Control Taskscited by 608
  10. 2019AI Feynman: a Physics-Inspired Method for Symbolic Regressioncited by 536
  11. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  12. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
22 more
  1. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  2. 2020Discovering Symbolic Models from Deep Learning with Inductive Biasescited by 335
  3. 2023Language Models Represent Space and Timecited by 306
  4. 2022Transformers Learn Shortcuts to Automatacited by 260
  5. 2022The Parallelism Tradeoff: Limitations of Log-Precision Transformerscited by 191
  6. 2024The Illusion of State in State-Space Modelscited by 181
  7. 2021Implicit Representations of Meaning in Neural Language Modelscited by 179
  8. 2019Predictive Multiplicity in Classificationcited by 162
  9. 2019A Meta-Transfer Objective for Learning to Disentangle Causal Mechanismscited by 158
  10. 2023Taken out of context: On measuring situational awareness in LLMscited by 131
  11. 2021Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Colorcited by 118
  12. 2018Discovering physical concepts with neural networkscited by 117
  13. 2024Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristicscited by 94
  14. 2022Physics-Informed Machine Learning: A Survey on Problems, Methods and Applicationscited by 88
  15. 2020AI Poincar'e: Machine Learning Conservation Laws from Trajectoriescited by 77
  16. 2022Rediscovering orbital mechanics with machine learningcited by 71
  17. 2021Chess as a Testbed for Language Model State Trackingcited by 69
  18. 2021Low-Complexity Probing via Finding Subnetworkscited by 63
  19. 2017Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physicscited by 57
  20. 2019The Information Complexity of Learning Tasks, their Structure and their Distancecited by 35
  21. 2023Red Teaming Deep Neural Networks with Feature Synthesis Toolscited by 21
  22. 2018On Evaluating the Generalization of LSTM Models in Formal Languagescited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 15; 2026: 31.

46 papers.

Sort:

By area:

  1. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  2. 2025Next-Latent Prediction Transformers Learn Compact World Modelscited by 29
  3. 2025The wall confronting large language modelscited by 8
  4. 2025When is a System Discoverable from Data? Discovery Requires Chaoscited by 6
  5. 2026From Kepler to Newton: Inductive Biases Guide Learned World Models in Transformerscited by 6
  6. 2025SkyDreamer: Interpretable End-to-End Vision-Based Drone Racing with Model-Based Reinforcement Learningcited by 5
  7. 2026Learning State-Tracking from Code Using Linear RNNscited by 5
  8. 2026From Observation to Insight: Mechanistic World Models and the Quest for Autonomous Discoverycited by 4
  9. 2026Overcoming Statistical Bias in Action-Controllable World Modelscited by 4
  10. 2025AI for Scientific Discovery is a Social Problemcited by 3
  11. 2026The Observer Effect in World Models: Invasive Adaptation Corrupts Latent Physicscited by 3
  12. 2026Dutch Books for Language Modelscited by 3
34 more
  1. 2025From Prediction to Understanding: Will AI Foundation Models Transform Brain Science?cited by 2
  2. 2025The Impossibility of Inverse Permutation Learning in Transformer Modelscited by 2
  3. 2026Learning Task-Sufficient World Models by Synergizing Agentic Exploration and Structured Modelingcited by 2
  4. 2025Adaptive Multi-Agent Reasoning via Automated Workflow Generationcited by 1
  5. 2025AI Needs Physics More Than Physics Needs AIcited by 1
  6. 2026Convergent World Representations and Divergent Taskscited by 1
  7. 2026SymTorch: Symbolic Distillation of Neural Networkscited by 1
  8. 2025Automated discovery of finite volume schemes using Graph Neural Networkscited by 0
  9. 2025Language models as tools for investigating the distinction between possible and impossible natural languagescited by 0
  10. 2025Large language models have learned to use languagecited by 0
  11. 2025Large language models are not about natural languagecited by 0
  12. 2025Linguists should learn to love speech-based deep learning modelscited by 0
  13. 2026Are Language Models Models?cited by 0
  14. 2026CHLU: The Causal Hamiltonian Learning Unit as a Symplectic Primitive for Deep Learningcited by 0
  15. 2026Across the Levels of Analysis: Explaining Predictive Processing in Humans Requires More Than Machine-Estimated Probabilitiescited by 0
  16. 2026You Can't Fight in Here! This is BBS!cited by 0
  17. 2026A paradox of AI fluencycited by 0
  18. 2026In-Context Examples Suppress Scientific Knowledge Recall in LLMscited by 0
  19. 2026Not-So-Strange Love: Language Models and Generative Linguistic Theories are More Compatible than They Appearcited by 0
  20. 2026GENSTRAT: Toward a Science of Strategic Reasoning in Large Language Modelscited by 0
  21. 2026Learning to Extrapolate to New Tasks: A Relational Approach to Task Extrapolationcited by 0
  22. 2026Two-Layer Linear Auto-Regressive Models Estimate Latent Statescited by 0
  23. 2026On the Smallness of the Large Language Models Scaling Exponentscited by 0
  24. 2026Learning Implicit Causal World Models from Multi-Agent Demonstrationscited by 0
  25. 2026LAWFUL: Law-Aligned Witness for Faithful Use of Latentscited by 0
  26. 2026Beyond Myopic World Models: Long-Horizon End-to-End Training for Direct Future Predictioncited by 0
  27. 2026Hierarchical Compositionality for An Assistive AI Agentcited by 0
  28. 2026OraclePhys: A Systematic Framework for LLM Fine-Tuning on Structural Mechanicscited by 0
  29. 2026Entity tracking emerges in sub-billion parameter language models and exceeds human performance in naturalistic narrativescited by 0
  30. 2026The Illusion of Replacement: Rethinking Specialized Machine Learning Models in the Foundation Model Eracited by 0
  31. 2026TetrisCNN for interpretable detection of phases of matter from experimental quantum simulator datacited by 0
  32. 2026World Modeling in Transformerscited by 0
  33. 2026PermuFormer: Multi-Task Pretraining for Permutation Representation in Algebraic Combinatoricscited by 0
  34. 2026MultiEcho: An Experimental Science of Learned Worldscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024On Foundation Models for Dynamical Systems from Purely Synthetic Data
  2. 2026NewtPhys: Do Foundation Models Understand Newtonian Physics?
  3. 2026CausalArena: Benchmarking Causal Discovery in the Foundation Model Era
  4. 2026Earth Science Foundation Models: From Perception to Reasoning and Discovery
  5. 2026Platonic Representation Hypothesis on World Models
  6. 2026Learning Spatio-Temporal Foundation Models from Pure Synthetic Data
  7. 2026ALDER: Discovering the Laws of a World by Acting in It
  8. 2025Aligning Inductive Bias for Data-Efficient Generalization in State Space Models