mlsanityData to 7 Oct 2026

A Comprehensive Survey on World Models for Embodied AI

arXiv 2510.16732 ↗First posted 19 Oct 2025cs.CVcited by 65

On the map: World models → World models for navigation

Abstract (arXiv)

Embodied AI requires agents that perceive, act, and anticipate how actions reshape future world states. World models serve as internal simulators that capture environment dynamics, enabling forward and counterfactual rollouts to support perception, prediction, and decision making. This survey presents a unified framework for world models in embodied AI. Specifically, we formalize the problem setting and learning objectives, and propose a three-axis taxonomy encompassing: (1) Functionality, Decision-Coupled vs. General-Purpose; (2) Temporal Modeling, Sequential Simulation and Inference vs. Global Difference Prediction; (3) Spatial Representation, Global Latent Vector, Token Feature Sequence, Spatial Latent Grid, and Decomposed Rendering Representation. We systematize data resources and metrics across robotics, autonomous driving, and general video settings, covering pixel prediction quality, state-level understanding, and task performance. Furthermore, we offer a quantitative comparison of state-of-the-art models and distill key open challenges, including the scarcity of unified datasets and the need for evaluation metrics that assess physical consistency over pixel fidelity, the trade-off between model performance and the computational efficiency required for real-time control, and the core modeling difficulty of achieving long-horizon temporal consistency while mitigating error accumulation. Finally, we maintain a curated bibliography at https://github.com/Li-Zn-H/AwesomeWorldModels.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Placed in World models for navigation, one of 11 directions in World models. Builds on 226 of its 260 references that are in the map. First posted on arXiv 19 Oct 2025.

Builds on

226 of 260 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025FOUNDER: Grounding Foundation Models in World Models for Open-Ended Embodied Decision Making15 Jul
  2. 2025AirScape: An Aerial Generative World Model with Motion Controllability10 Jul
  3. 2025NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environments30 Jun
  4. 2025Embodied AI Agents: Modeling the World27 Jun
  5. 2025World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning13 Mar
  6. 2025WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigation4 Mar
  7. 2024Navigation World Models4 Dec
  8. 2024X-MOBILITY: End-To-End Generalizable Navigation via World Modeling23 Oct
  9. 2024The Essential Role of Causality in Foundation World Models for Embodied AI6 Feb
  10. 2019Habitat: A Platform for Embodied AI Research2 Apr

Other references · 216

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  6. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  7. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  8. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  9. 2017The Kinetics Human Action Video Datasetcited by 1,894
  10. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  11. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  12. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
38 more
  1. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  2. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  3. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  4. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  5. 2022Planning-oriented Autonomous Drivingcited by 1,068
  6. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  7. 2020Mastering Atari with Discrete World Modelscited by 990
  8. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  9. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  10. 2018DeepMind Control Suitecited by 935
  11. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  12. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
  13. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  14. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  15. 2024Genie: Generative Interactive Environmentscited by 706
  16. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  17. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  18. 2019Model-Based Reinforcement Learning for Ataricited by 495
  19. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
  20. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
  21. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  22. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 357
  23. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
  24. 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 340
  25. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  26. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  27. 2022Transformers are Sample-Efficient World Modelscited by 317
  28. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
  29. 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 308
  30. 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 239
  31. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 230
  32. 2024GenAD: Generative End-to-End Autonomous Drivingcited by 225
  33. 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasetscited by 218
  34. 2020Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Predictioncited by 192
  35. 2024RoboDreamer: Learning Compositional World Models for Robot Imaginationcited by 187
  36. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 186
  37. 2022MERLOT Reserve: Neural Script Knowledge through Vision and Language and Soundcited by 184
  38. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175

50 most cited of 216 All 216 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 63.

65 papers.

Sort:

By area:

  1. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  2. 2026RISE: Self-Improving Robot Policy with Compositional World Modelcited by 39
  3. 2026Interpreting Physics in Video World Modelscited by 18
  4. 2026Flash-WAM: Modality-Aware Distillation for World Action Modelscited by 16
  5. 2026Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directionscited by 12
  6. 2026Wow, wo, val! A Comprehensive Embodied World Model Evaluation Turing Testcited by 11
  7. 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 10
  8. 2026STARRY: Spatial-Temporal Action-Centric World Modeling for Robotic Manipulationcited by 10
  9. 2026CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Drivingcited by 10
  10. 2026RAE-NWM: Navigation World Model in Dense Visual Representation Spacecited by 9
  11. 2026Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Drivingcited by 9
  12. 2026ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulationcited by 8
38 more
  1. 2026WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platformcited by 8
  2. 2026WM-DAgger: Enabling Efficient Data Aggregation for Imitation Learning with World Modelscited by 7
  3. 2026The DAWN of World-Action Interactive Modelscited by 7
  4. 2026Point Tracking Improves World Action Modelscited by 7
  5. 2026H-WM: Robotic Task and Motion Planning Guided by Hierarchical World Modelcited by 6
  6. 2026Embodied Foundation Models at the Edge: A Survey of Deployment Constraints and Mitigation Strategiescited by 6
  7. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
  8. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Datacited by 5
  9. 2026VLA-Thinker: Boosting Vision-Language-Action Models through Thinking-with-Image Reasoningcited by 5
  10. 2026Human Cognition in Machines: A Unified Perspective of World Modelscited by 5
  11. 2026Self-Evolving World Models for LLM Agent Planningcited by 5
  12. 2025Counterfactual World Models via Digital Twin-conditioned Video Diffusioncited by 4
  13. 2026Echo-Memory: A Controlled Study of Memory in Action World Modelscited by 4
  14. 2026Safety, Security, and Cognitive Risks in World Modelscited by 3
  15. 2026OpenWorldLib: A Unified Codebase and Definition of Advanced World Modelscited by 3
  16. 2026Toward World Models for Epidemiologycited by 3
  17. 2026PhyWorld: Physics-Faithful World Model for Video Generationcited by 3
  18. 2026World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applicationscited by 3
  19. 2026Bridging the Agent-World Gap: Text World Models for LLM-based Agentscited by 3
  20. 2026BadWorld: Adversarial Attacks on World Modelscited by 3
  21. 2026Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Modelcited by 3
  22. 2026Medical world models in healthcare: foundations, applications, and challenges for trustworthy clinical translationcited by 3
  23. 2026The Semantic Lifecycle in Embodied AI: Acquisition, Representation and Storage via Foundation Modelscited by 2
  24. 2026An Empirical Study of World Model Quantizationcited by 2
  25. 2026WorldCache: Content-Aware Caching for Accelerated Video World Modelscited by 2
  26. 2026Looped World Modelscited by 2
  27. 2026Pondering the Way: Spatial-perceiving World Action Model for Embodied Navigationcited by 2
  28. 2025Embodied Tree of Thoughts: Deliberate Manipulation Planning with Embodied World Modelcited by 1
  29. 2026Rotation-Invariant Spherical Watermarking via Third-Order SO(3) Representation Couplingcited by 1
  30. 2026BadDreamer: Transferable Backdoor Attacks against Video World Models for Autonomous Drivingcited by 1
  31. 2026CosmosAlign: Adapting a World Foundation Model for Generative Traffic Video Forecastingcited by 1
  32. 2026RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstructioncited by 1
  33. 2026World Models for Embodied Intelligence: From Plausible to Controllable to Actionablecited by 1
  34. 2026From Digital Twins to World Models:Opportunities, Challenges, and Applications for Mobile Edge General Intelligencecited by 0
  35. 2026Event-Centric World Modeling with Memory-Augmented Retrieval for Embodied Decision-Makingcited by 0
  36. 2026OmniJigsaw: Enhancing Omni-Modal Reasoning via Modality-Orchestrated Reorderingcited by 0
  37. 2026Render, Don't Decode: Weight-Space World Models with Latent Structural Disentanglementcited by 0
  38. 2026PhysMani: Physics-principled 3D World Model for Dynamic Object Manipulationcited by 0

50 most cited of 65 All 65 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025World-in-World: World Models in a Closed-Loop World
  2. 2026How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Position
  3. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Models
  4. 2026World Model for Robot Learning: A Comprehensive Survey
  5. 2026WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models
  6. 2025The Safety Challenge of World Models for Embodied AI Agents: A Review
  7. 2026Physically Native World Models: A Hamiltonian Perspective on Generative World Modeling
  8. 2026Do World Models Make Better Robots? A Survey of Evaluation Benchmarks for Predictive Embodied Intelligence