A Comprehensive Survey on World Models for Embodied AI
On the map: World models → World models for navigation
Abstract (arXiv)
Embodied AI requires agents that perceive, act, and anticipate how actions reshape future world states. World models serve as internal simulators that capture environment dynamics, enabling forward and counterfactual rollouts to support perception, prediction, and decision making. This survey presents a unified framework for world models in embodied AI. Specifically, we formalize the problem setting and learning objectives, and propose a three-axis taxonomy encompassing: (1) Functionality, Decision-Coupled vs. General-Purpose; (2) Temporal Modeling, Sequential Simulation and Inference vs. Global Difference Prediction; (3) Spatial Representation, Global Latent Vector, Token Feature Sequence, Spatial Latent Grid, and Decomposed Rendering Representation. We systematize data resources and metrics across robotics, autonomous driving, and general video settings, covering pixel prediction quality, state-level understanding, and task performance. Furthermore, we offer a quantitative comparison of state-of-the-art models and distill key open challenges, including the scarcity of unified datasets and the need for evaluation metrics that assess physical consistency over pixel fidelity, the trade-off between model performance and the computational efficiency required for real-time control, and the core modeling difficulty of achieving long-horizon temporal consistency while mitigating error accumulation. Finally, we maintain a curated bibliography at https://github.com/Li-Zn-H/AwesomeWorldModels.
Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Placed in World models for navigation, one of 11 directions in World models. Builds on 226 of its 260 references that are in the map. First posted on arXiv 19 Oct 2025.
Builds on
226 of 260 references.
Closest ancestors · 10
Same direction, newest first
- 2025FOUNDER: Grounding Foundation Models in World Models for Open-Ended Embodied Decision Making15 Jul
- 2025AirScape: An Aerial Generative World Model with Motion Controllability10 Jul
- 2025NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environments30 Jun
- 2025Embodied AI Agents: Modeling the World27 Jun
- 2025World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning13 Mar
- 2025WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigation4 Mar
- 2024Navigation World Models4 Dec
- 2024X-MOBILITY: End-To-End Generalizable Navigation via World Modeling23 Oct
- 2024The Essential Role of Causality in Foundation World Models for Embodied AI6 Feb
- 2019Habitat: A Platform for Embodied AI Research2 Apr
Other references · 216
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
- 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
- 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
- 2017The Kinetics Human Action Video Datasetcited by 1,894
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
38 more
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
- 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
- 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
- 2022Planning-oriented Autonomous Drivingcited by 1,068
- 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
- 2020Mastering Atari with Discrete World Modelscited by 990
- 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
- 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
- 2018DeepMind Control Suitecited by 935
- 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
- 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
- 2024Genie: Generative Interactive Environmentscited by 706
- 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
- 2019Model-Based Reinforcement Learning for Ataricited by 495
- 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
- 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
- 2022DayDreamer: World Models for Physical Robot Learningcited by 372
- 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 357
- 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
- 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 340
- 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
- 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
- 2022Transformers are Sample-Efficient World Modelscited by 317
- 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
- 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 308
- 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 239
- 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 230
- 2024GenAD: Generative End-to-End Autonomous Drivingcited by 225
- 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasetscited by 218
- 2020Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Predictioncited by 192
- 2024RoboDreamer: Learning Compositional World Models for Robot Imaginationcited by 187
- 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 186
- 2022MERLOT Reserve: Neural Script Knowledge through Vision and Language and Soundcited by 184
- 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
50 most cited of 216 All 216 (JSON)
Built on by
Citing papers per year: 2025: 2; 2026: 63.
65 papers.
Sort:
By area:
- 2026World Action Models: The Next Frontier in Embodied AIcited by 46
- 2026RISE: Self-Improving Robot Policy with Compositional World Modelcited by 39
- 2026Interpreting Physics in Video World Modelscited by 18
- 2026Flash-WAM: Modality-Aware Distillation for World Action Modelscited by 16
- 2026Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directionscited by 12
- 2026Wow, wo, val! A Comprehensive Embodied World Model Evaluation Turing Testcited by 11
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 10
- 2026STARRY: Spatial-Temporal Action-Centric World Modeling for Robotic Manipulationcited by 10
- 2026CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Drivingcited by 10
- 2026RAE-NWM: Navigation World Model in Dense Visual Representation Spacecited by 9
- 2026Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Drivingcited by 9
- 2026ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulationcited by 8
38 more
- 2026WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platformcited by 8
- 2026WM-DAgger: Enabling Efficient Data Aggregation for Imitation Learning with World Modelscited by 7
- 2026The DAWN of World-Action Interactive Modelscited by 7
- 2026Point Tracking Improves World Action Modelscited by 7
- 2026H-WM: Robotic Task and Motion Planning Guided by Hierarchical World Modelcited by 6
- 2026Embodied Foundation Models at the Edge: A Survey of Deployment Constraints and Mitigation Strategiescited by 6
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
- 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Datacited by 5
- 2026VLA-Thinker: Boosting Vision-Language-Action Models through Thinking-with-Image Reasoningcited by 5
- 2026Human Cognition in Machines: A Unified Perspective of World Modelscited by 5
- 2026Self-Evolving World Models for LLM Agent Planningcited by 5
- 2025Counterfactual World Models via Digital Twin-conditioned Video Diffusioncited by 4
- 2026Echo-Memory: A Controlled Study of Memory in Action World Modelscited by 4
- 2026Safety, Security, and Cognitive Risks in World Modelscited by 3
- 2026OpenWorldLib: A Unified Codebase and Definition of Advanced World Modelscited by 3
- 2026Toward World Models for Epidemiologycited by 3
- 2026PhyWorld: Physics-Faithful World Model for Video Generationcited by 3
- 2026World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applicationscited by 3
- 2026Bridging the Agent-World Gap: Text World Models for LLM-based Agentscited by 3
- 2026BadWorld: Adversarial Attacks on World Modelscited by 3
- 2026Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Modelcited by 3
- 2026Medical world models in healthcare: foundations, applications, and challenges for trustworthy clinical translationcited by 3
- 2026The Semantic Lifecycle in Embodied AI: Acquisition, Representation and Storage via Foundation Modelscited by 2
- 2026An Empirical Study of World Model Quantizationcited by 2
- 2026WorldCache: Content-Aware Caching for Accelerated Video World Modelscited by 2
- 2026Looped World Modelscited by 2
- 2026Pondering the Way: Spatial-perceiving World Action Model for Embodied Navigationcited by 2
- 2025Embodied Tree of Thoughts: Deliberate Manipulation Planning with Embodied World Modelcited by 1
- 2026Rotation-Invariant Spherical Watermarking via Third-Order SO(3) Representation Couplingcited by 1
- 2026BadDreamer: Transferable Backdoor Attacks against Video World Models for Autonomous Drivingcited by 1
- 2026CosmosAlign: Adapting a World Foundation Model for Generative Traffic Video Forecastingcited by 1
- 2026RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstructioncited by 1
- 2026World Models for Embodied Intelligence: From Plausible to Controllable to Actionablecited by 1
- 2026From Digital Twins to World Models:Opportunities, Challenges, and Applications for Mobile Edge General Intelligencecited by 0
- 2026Event-Centric World Modeling with Memory-Augmented Retrieval for Embodied Decision-Makingcited by 0
- 2026OmniJigsaw: Enhancing Omni-Modal Reasoning via Modality-Orchestrated Reorderingcited by 0
- 2026Render, Don't Decode: Weight-Space World Models with Latent Structural Disentanglementcited by 0
- 2026PhysMani: Physics-principled 3D World Model for Dynamic Object Manipulationcited by 0
50 most cited of 65 All 65 (JSON)
- 2026DreamTrue: Action-Faithful Robot World Model with Counterfactual Post-Training8 Oct 2026
- 2026RoboCoach: World Models as Active Coaches for Compositional Robot Skills30 Sep 2026
- 2026HappyWorld-Bench21 Sep 2026
- 2026UniK: Universal Knowledge Perception for Digital and Physical AI21 Sep 2026
- 2026A Comprehensive Review of Generative Physical Artificial Intelligence16 Sep 2026
- 2026World Models for Embodied Intelligence: From Plausible to Controllable to Actionable15 Sep 2026
- 2026Toward Unified Robot Learning: Bridging Representation, Vision-Language-Action, and World Models3 Sep 2026
- 2026World Model-Guided Reinforcement Learning via Counterfactual User Engagement Simulation1 Sep 2026
- 2026RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstruction28 Aug 2026
- 2026WM-R1: Training GUI Agents to Reason and leverage World Models with Reinforcement Learning27 Aug 2026
- 2026StreamPI: Streaming Multimodal Temporal Modeling for Vision-Language-Action Models26 Aug 2026
- 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?24 Aug 2026
38 more
- 2026Toward the Cognitive--Physical Limits of Embodied Intelligence through a World-Model-Centric Autonomous Racing Agent11 Aug 2026
- 2026CosmosAlign: Adapting a World Foundation Model for Generative Traffic Video Forecasting7 Aug 2026
- 2026Quo Vadis, World Modeling?3 Aug 2026
- 2026Medical world models in healthcare: foundations, applications, and challenges for trustworthy clinical translation28 Jul 2026
- 2026LeapBot-WA: World-Anchor Action Models via Predictive Latent Alignments27 Jul 2026
- 2026RoboInter1.5: A Holistic Intermediate Representation Suite for Embodied World Modeling and Robotic Manipulation21 Jul 2026
- 2026Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model13 Jul 2026
- 2026PhysMani: Physics-principled 3D World Model for Dynamic Object Manipulation2 Jul 2026
- 2026Self-Evolving World Models for LLM Agent Planning29 Jun 2026
- 2026Pondering the Way: Spatial-perceiving World Action Model for Embodied Navigation29 Jun 2026
- 2026BadDreamer: Transferable Backdoor Attacks against Video World Models for Autonomous Driving19 Jun 2026
- 2026Looped World Models16 Jun 2026
- 2026BadWorld: Adversarial Attacks on World Models15 Jun 2026
- 2026Echo-Memory: A Controlled Study of Memory in Action World Models8 Jun 2026
- 2026Bridging the Agent-World Gap: Text World Models for LLM-based Agents8 Jun 2026
- 2026Flash-WAM: Modality-Aware Distillation for World Action Models3 Jun 2026
- 2026WMLLM: Self-Evolving Optimization Agents via Predict-Then-Act World Modeling2 Jun 2026
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends31 May 2026
- 2026World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applications28 May 2026
- 2026Rotation-Invariant Spherical Watermarking via Third-Order SO(3) Representation Coupling26 May 2026
- 2026Point Tracking Improves World Action Models22 May 2026
- 2026PhyWorld: Physics-Faithful World Model for Video Generation19 May 2026
- 2026WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platform18 May 2026
- 2026World Action Models: The Next Frontier in Embodied AI12 May 2026
- 2026The DAWN of World-Action Interactive Models12 May 2026
- 2026CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Driving11 May 2026
- 2026Render, Don't Decode: Weight-Space World Models with Latent Structural Disentanglement7 May 2026
- 2026STARRY: Spatial-Temporal Action-Centric World Modeling for Robotic Manipulation29 Apr 2026
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond24 Apr 2026
- 2026Human Cognition in Machines: A Unified Perspective of World Models17 Apr 2026
- 2026WM-DAgger: Enabling Efficient Data Aggregation for Imitation Learning with World Models13 Apr 2026
- 2026Toward World Models for Epidemiology10 Apr 2026
- 2026OmniJigsaw: Enhancing Omni-Modal Reasoning via Modality-Orchestrated Reordering9 Apr 2026
- 2026Event-Centric World Modeling with Memory-Augmented Retrieval for Embodied Decision-Making8 Apr 2026
- 2026OpenWorldLib: A Unified Codebase and Definition of Advanced World Models6 Apr 2026
- 2026Safety, Security, and Cognitive Risks in World Models1 Apr 2026
- 2026ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulation29 Mar 2026
- 2026Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Driving28 Mar 2026
50 newest of 65 All 65 (JSON)
- 2026World Action Models: The Next Frontier in Embodied AIcited by 46
- 2026RISE: Self-Improving Robot Policy with Compositional World Modelcited by 39
- 2026Interpreting Physics in Video World Modelscited by 18
- 2026Flash-WAM: Modality-Aware Distillation for World Action Modelscited by 16
- 2026Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directionscited by 12
- 2026Wow, wo, val! A Comprehensive Embodied World Model Evaluation Turing Testcited by 11
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 10
- 2026STARRY: Spatial-Temporal Action-Centric World Modeling for Robotic Manipulationcited by 10
- 2026CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Drivingcited by 10
- 2026RAE-NWM: Navigation World Model in Dense Visual Representation Spacecited by 9
- 2026Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Drivingcited by 9
- 2026WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platformcited by 8
12 most cited of 53 All 53 (JSON)
- 2026RoboCoach: World Models as Active Coaches for Compositional Robot Skills30 Sep 2026
- 2026HappyWorld-Bench21 Sep 2026
- 2026World Models for Embodied Intelligence: From Plausible to Controllable to Actionable15 Sep 2026
- 2026Toward Unified Robot Learning: Bridging Representation, Vision-Language-Action, and World Models3 Sep 2026
- 2026World Model-Guided Reinforcement Learning via Counterfactual User Engagement Simulation1 Sep 2026
- 2026RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstruction28 Aug 2026
- 2026WM-R1: Training GUI Agents to Reason and leverage World Models with Reinforcement Learning27 Aug 2026
- 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?24 Aug 2026
- 2026Toward the Cognitive--Physical Limits of Embodied Intelligence through a World-Model-Centric Autonomous Racing Agent11 Aug 2026
- 2026Quo Vadis, World Modeling?3 Aug 2026
- 2026Medical world models in healthcare: foundations, applications, and challenges for trustworthy clinical translation28 Jul 2026
- 2026LeapBot-WA: World-Anchor Action Models via Predictive Latent Alignments27 Jul 2026
12 newest of 53 All 53 (JSON)
- 2026Embodied Foundation Models at the Edge: A Survey of Deployment Constraints and Mitigation Strategiescited by 6
- 2026The Semantic Lifecycle in Embodied AI: Acquisition, Representation and Storage via Foundation Modelscited by 2
- 2026Rotation-Invariant Spherical Watermarking via Third-Order SO(3) Representation Couplingcited by 1
- 2026CosmosAlign: Adapting a World Foundation Model for Generative Traffic Video Forecastingcited by 1
- 2026OmniJigsaw: Enhancing Omni-Modal Reasoning via Modality-Orchestrated Reorderingcited by 0
- 2026UniK: Universal Knowledge Perception for Digital and Physical AIcited by 0
- 2026DreamTrue: Action-Faithful Robot World Model with Counterfactual Post-Trainingcited by 0
- 2026DreamTrue: Action-Faithful Robot World Model with Counterfactual Post-Training8 Oct 2026
- 2026UniK: Universal Knowledge Perception for Digital and Physical AI21 Sep 2026
- 2026CosmosAlign: Adapting a World Foundation Model for Generative Traffic Video Forecasting7 Aug 2026
- 2026Rotation-Invariant Spherical Watermarking via Third-Order SO(3) Representation Coupling26 May 2026
- 2026OmniJigsaw: Enhancing Omni-Modal Reasoning via Modality-Orchestrated Reordering9 Apr 2026
- 2026Embodied Foundation Models at the Edge: A Survey of Deployment Constraints and Mitigation Strategies16 Mar 2026
- 2026The Semantic Lifecycle in Embodied AI: Acquisition, Representation and Storage via Foundation Models12 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2025World-in-World: World Models in a Closed-Loop World
- 2026How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Position
- 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Models
- 2026World Model for Robot Learning: A Comprehensive Survey
- 2026WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models
- 2025The Safety Challenge of World Models for Embodied AI Agents: A Review
- 2026Physically Native World Models: A Hamiltonian Perspective on Generative World Modeling
- 2026Do World Models Make Better Robots? A Survey of Evaluation Benchmarks for Predictive Embodied Intelligence