World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applications
On the map: World models → World models for language agents
Abstract (arXiv)
World models, internal simulators that learn the structure and dynamics of an environment, have emerged as a central paradigm in the pursuit of artificial general intelligence, enabling agents to predict, plan, and reason within learned representations. Despite rapid progress across reinforcement learning, robotics, autonomous driving, and video generation, the field lacks a unified framework integrating its diverse architectural choices, training methods, reasoning mechanisms, and application settings. This survey addresses that gap with a multi-axis taxonomy organized along four dimensions: (i) architecture, encompassing representation format, dynamics formulation, input modality, learning paradigm, and downstream application; (ii) methodological family, including state-space and recurrent approaches, transformer-based models, diffusion-based generators, physics-informed networks, and language-augmented multimodal systems; (iii) reasoning strategy, covering imagination-based planning, latent policy learning, counterfactual reasoning, and planning under uncertainty; and (iv) application domain, spanning robotics, autonomous driving, video prediction, multimodal agents, reinforcement learning, scientific modeling, medical imaging, educational measurement, and business and finance. Tracing the field from early cognitive-science foundations to milestone systems such as PlaNet, the Dreamer family, MuZero, Sora, Cosmos, and Genie, we examine how these dimensions interact and highlight the recent convergence of chain-of-thought reasoning with world-model imagination. We review evaluation protocols and benchmarks, identify persistent challenges such as compounding prediction errors, sim-to-real transfer, and fragmented evaluation, and outline future directions toward unified multimodal world models, foundation-scale interactive simulators, and safe deployment in safety-critical domains.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 134 of its 430 references that are in the map. First posted on arXiv 28 May 2026.
Builds on
134 of 430 references.
Closest ancestors · 2
Same direction, newest first
- 2024Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents10 Nov
- 2024From Efficient Multimodal Models to World Models: A Survey27 Jun
Other references · 132
- 2017Attention Is All You Needcited by 60,361
- 2023GPT-4 Technical Reportcited by 18,396
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
- 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
- 2019Invariant Risk Minimizationcited by 1,854
- 2023Consistency Modelscited by 1,822
38 more
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
- 2018World Modelscited by 1,513
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
- 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
- 2018Relational inductive biases, deep learning, and graph networkscited by 1,360
- 2023Mastering Diverse Domains through World Modelscited by 1,253
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
- 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
- 2018DeepMind Control Suitecited by 956
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
- 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 846
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
- 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
- 2024Genie: Generative Interactive Environmentscited by 747
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
- 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
- 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 652
- 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
- 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
- 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
- 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
- 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
- 2019Model-Based Reinforcement Learning for Ataricited by 502
- 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 495
- 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 493
- 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
- 2020Lagrangian Neural Networkscited by 398
- 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 386
- 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 315
- 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 300
- 2024Diffusion Models Are Real-Time Game Enginescited by 296
- 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 269
- 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 252
- 2025Motus: A Unified Latent Action World Modelcited by 251
- 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
50 most cited of 132 All 132 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026World Model for Robot Learning: A Comprehensive Survey
- 2025PAN: A World Model for General, Actionable, and Long-Horizon World Simulation
- 2026A Tutorial on World Models and Physical AI
- 2026World Reasoning Arena
- 2026From Generative Engines to Actionable Simulators: The Imperative of Physical Grounding in World Models
- 2026World-Action Models for Robot Learning and Control: A Survey
- 2026World Models as an Intermediary between Agents and the Real World
- 2026World Action Models: The Next Frontier in Embodied AI