# General agents contain world models

arXiv: https://arxiv.org/abs/2506.01622 · HTML: https://mlsanity.com/paper/2506.01622/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in World models for language agents, one of 11 directions in World models.
- Builds on 51 of its 98 references that are in the map.
- First posted on arXiv 2 Jun 2025.

## Placed in

- World models / [World models for language agents](https://mlsanity.com/wm/world-models-for-language-agents/)

## Builds on (most cited first, 30 of 51)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [Reasoning with Language Model is Planning with World Model](https://mlsanity.com/paper/2305.14992/) (2023)
- [A Generalist Agent](https://mlsanity.com/paper/2205.06175/) (2022)
- [Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models](https://mlsanity.com/paper/1805.12114/) (2018)
- [Scalable agent alignment via reward modeling: a research direction](https://mlsanity.com/paper/1811.07871/) (2018)
- [Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task](https://mlsanity.com/paper/2210.13382/) (2022)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [Language Models Represent Space and Time](https://mlsanity.com/paper/2310.02207/) (2023)
- [Challenges of Real-World Reinforcement Learning](https://mlsanity.com/paper/1904.12901/) (2019)
- [Transfer Learning in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2009.07888/) (2020)
- [Machine Theory of Mind](https://mlsanity.com/paper/1802.07740/) (2018)
- [Goal-Conditioned Reinforcement Learning: Problems and Solutions](https://mlsanity.com/paper/2201.08299/) (2022)
- [Imagination-Augmented Agents for Deep Reinforcement Learning](https://mlsanity.com/paper/1707.06203/) (2017)
- [Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color](https://mlsanity.com/paper/2109.06129/) (2021)
- [Reinforcement Learning With Temporal Logic Rewards](https://mlsanity.com/paper/1612.03471/) (2016)
- [Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems](https://mlsanity.com/paper/2405.06624/) (2024)
- [Robust agents learn causal world models](https://mlsanity.com/paper/2402.10877/) (2024)
- [Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guarantees](https://mlsanity.com/paper/1909.05304/) (2019)
- [Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?](https://mlsanity.com/paper/2502.15657/) (2025)

## Built on by (most cited first, 17 of 17)

- [Do World Action Models Generalize Better than VLAs? A Robustness Study](https://mlsanity.com/paper/2603.22078/) (2026)
- [Qwen-AgentWorld: Language World Models for General Agents](https://mlsanity.com/paper/2606.24597/) (2026)
- [Assessing Adaptive World Models in Machines with Novel Games](https://mlsanity.com/paper/2507.12821/) (2025)
- [Self-Predictive Representations for Combinatorial Generalization in Behavioral Cloning](https://mlsanity.com/paper/2506.10137/) (2025)
- [Cogito, Ergo Ludo: An Agent that Learns to Play by Reasoning and Planning](https://mlsanity.com/paper/2509.25052/) (2025)
- [Nano World Models: A Minimalist Implementation of Future Video Prediction](https://mlsanity.com/paper/2605.23993/) (2026)
- [A "good regulator theorem" for embodied agents](https://mlsanity.com/paper/2508.06326/) (2025)
- [Generalizable Dense Reward for Long-Horizon Robotic Tasks](https://mlsanity.com/paper/2604.00055/) (2026)
- [Affordances Enable Partial World Modeling with LLMs](https://mlsanity.com/paper/2602.10390/) (2026)
- [What Capable Agents Must Know: Selection Theorems for Robust Decision-Making under Uncertainty](https://mlsanity.com/paper/2603.02491/) (2026)
- [Epistemic Traps: Rational Misalignment Driven by Model Misspecification](https://mlsanity.com/paper/2602.17676/) (2026)
- [Event-Driven Reinforcement Learning Enables Long-Horizon Control in Semiconductor Fabrication](https://mlsanity.com/paper/2606.10705/) (2026)
- [From Optimal Actions to World Models: Identifiability of Transition Kernels in Discounted MDPs](https://mlsanity.com/paper/2608.07301/) (2026)
- [The syntax and semantics of goals](https://mlsanity.com/paper/2609.19448/) (2026)
- [DSWM: Decomposed Spatio-Temporal World Model for Demand-Driven UAV Base Station Repositioning](https://mlsanity.com/paper/2609.36845/) (2026)
- [Software World Models: From Consequence Prediction to Decision Value](https://mlsanity.com/paper/2610.04940/) (2026)
- [The Anatomy of a Blind Spot: What an LLM Judge Must Be Shown, and How Much](https://mlsanity.com/paper/2610.15938/) (2026)

## Abstract (arXiv)

Are world models a necessary ingredient for flexible, goal-directed behaviour, or is model-free learning sufficient? We provide a formal answer to this question, showing that any agent capable of generalizing to multi-step goal-directed tasks must have learned a predictive model of its environment. We show that this model can be extracted from the agent's policy, and that increasing the agents performance or the complexity of the goals it can achieve requires learning increasingly accurate world models. This has a number of consequences: from developing safe and general agents, to bounding agent capabilities in complex environments, and providing new algorithms for eliciting world models from agents.