From Observation to Insight: Mechanistic World Models and the Quest for Autonomous Discovery
On the map: World models → World models for language agents
Abstract (arXiv)
Recent advances in foundation models have transformed AI for Science, enabling remarkably accurate predictive performance across domains ranging from protein folding to weather forecasting. Yet prediction alone does not constitute scientific discovery. Scientific understanding depends on uncovering the reusable explanatory mechanisms that generate observations, whereas contemporary machine learning remains fundamentally organised around predictive mappings rather than explanatory structure. In this paper, we argue that scientific discovery is fundamentally a problem of knowledge organisation. To this end, we introduce Mechanistic World Models, a new design paradigm that places reusable mechanisms at the centre of representation, computation and learning. Drawing on insights from the philosophy of science, we derive the computational capabilities required for discovery, identify the design principles and inductive pressures that encourage explanatory knowledge to emerge, and formalise the anatomy of a mechanism-centric world model. Finally, we show how diverse research directions including mechanistic interpretability, causal representation learning, equation discovery and modular architectures capture complementary ingredients of this paradigm while lacking a unified framework. We propose Mechanistic World Models as a conceptual foundation and computational blueprint for moving AI beyond predictive forecasting towards autonomous scientific discovery.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 41 of its 85 references that are in the map. First posted on arXiv 14 Jul 2026.
Builds on
41 of 85 references.
Closest ancestors · 4
Same direction, newest first
- 2025What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models9 Jul
- 2025PoE-World: Compositional World Modeling with Products of Programmatic Experts16 May
- 2024Compete and Compose: Learning Independent Mechanisms for Modular World Models23 Apr
- 2022Variational Causal Dynamics: Discovering Modular World Models from Interventions22 Jun
Other references · 37
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
- 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
- 2020Object-Centric Learning with Slot Attentioncited by 865
- 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
- 2019AI Feynman: a Physics-Inspired Method for Symbolic Regressioncited by 536
- 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
- 2025Accelerating scientific discovery with Co-Scientistcited by 353
- 2020Discovering Symbolic Models from Deep Learning with Inductive Biasescited by 335
- 2019Causality for Machine Learningcited by 242
25 more
- 2019Recurrent Independent Mechanismscited by 219
- 2021Contemporary Symbolic Regression Methods and their Relative Performancecited by 209
- 2019Contrastive Learning of Structured World Modelscited by 206
- 2024Transcoders Find Interpretable LLM Feature Circuitscited by 186
- 2019Causal Discovery from Heterogeneous/Nonstationary Data with Independent Changescited by 166
- 2020DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learningcited by 159
- 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 158
- 2026Towards End-to-End Automation of AI Researchcited by 150
- 2022CITRIS: Causal Identifiability from Temporal Intervened Sequencescited by 132
- 2021GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinementcited by 109
- 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 105
- 2017Learning Independent Causal Mechanismscited by 104
- 2024InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoderscited by 64
- 2024Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learningcited by 56
- 2025FourCastNet 3: A geometric approach to probabilistic machine-learning weather forecasting at scalecited by 56
- 2021Conditional Neural Relational Inference for Interacting Systemscited by 28
- 2026Causal-JEPA: Learning World Models through Object-Level Latent Maskingcited by 25
- 2024Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description lengthcited by 16
- 2024SPARTAN: A Sparse Transformer World Model Attending to What Matterscited by 10
- 2026Systematic Evaluation of Single-Cell Foundation Model Interpretability Reveals Attention Captures Co-Expression Rather Than Unique Regulatory Signalcited by 7
- 2025Intrinsically Interpretable Attention via Sparse Post-Trainingcited by 4
- 2026Disentangling Dynamical Systems: Causal Representation Learning Meets Local Sparse Attentioncited by 3
- 2025Prediction is not Explanation: Revisiting the Explanatory Capacity of Mapping Embeddingscited by 2
- 2026Learning to Theorize the World from Observationcited by 2
- 2024The Use of AI-Robotic Systems for Scientific Discoverycited by 1
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world modelscited by 6
- 2026SJEPA: Learning Elegant Latent Dynamics with Hybrid Symbolic-Neural Predictorscited by 0
- 2026Benchmarking World Models for Continual Learning on Compositional Taskscited by 0
- 2026LLM-IDEA: Identifiability-Driven Experimental Agent for Autonomous Discovery of Mechanistic World Modelscited by 0
- 2026LLM-IDEA: Identifiability-Driven Experimental Agent for Autonomous Discovery of Mechanistic World Models8 Oct 2026
- 2026Benchmarking World Models for Continual Learning on Compositional Tasks18 Sep 2026
- 2026Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models10 Aug 2026
- 2026SJEPA: Learning Elegant Latent Dynamics with Hybrid Symbolic-Neural Predictors4 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence
- 2026MechBench: Can AI Scientific Agents Discover Mechanisms Beyond Phenomenal Laws?
- 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery
- 2025Virtuous Machines: Towards Artificial General Science
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
- 2024Towards Scientific Discovery with Generative AI: Progress, Opportunities, and Challenges
- 2025Scaling Laws in Scientific Discovery with AI and Robot Scientists
- 2026A Probabilistic Framework for LLM-Based Model Discovery