mlsanityData to 14 Oct 2026

The Safety Challenge of World Models for Embodied AI Agents: A Review

arXiv 2510.05865 ↗First posted 7 Oct 2025cs.AI cs.CV cs.ROcited by 6

On the map: World models → World models for navigation

Abstract (arXiv)

The rapid progress in embodied artificial intelligence has highlighted the necessity for more advanced and integrated models that can perceive, interpret, and predict environmental dynamics. In this context, World Models (WMs) have been introduced to provide embodied agents with the abilities to anticipate future environmental states and fill in knowledge gaps, thereby enhancing agents' ability to plan and execute actions. However, when dealing with embodied agents it is fundamental to ensure that predictions are safe for both the agent and the environment. In this article, we conduct a comprehensive literature review of World Models in the domains of autonomous driving and robotics, with a specific focus on the safety implications of scene and control generation tasks. Our review is complemented by an empirical analysis, wherein we collect and examine predictions from state-of-the-art models, identify and categorize common faults (herein referred to as pathologies), and provide a quantitative evaluation of the results.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in World models for navigation, one of 11 directions in World models. Builds on 54 of its 65 references that are in the map. First posted on arXiv 7 Oct 2025.

Builds on

54 of 65 references.

References · 54

  1. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  2. 2022Scalable Diffusion Models with Transformerscited by 6,054
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  6. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  7. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  8. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  9. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  10. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  11. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  12. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,304
38 more
  1. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  2. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  3. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  4. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  5. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  6. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
  7. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
  8. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
  9. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  10. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 523
  11. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 495
  12. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 347
  13. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
  14. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 319
  15. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
  16. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 261
  17. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 252
  18. 2024GenAD: Generative End-to-End Autonomous Drivingcited by 225
  19. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 218
  20. 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 197
  21. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 191
  22. 2024Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generationcited by 174
  23. 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 168
  24. 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 164
  25. 2022Model-Based Imitation Learning for Urban Drivingcited by 150
  26. 2024The Revolution of Multimodal Large Language Models: A Surveycited by 137
  27. 2023Language-Guided Traffic Simulation via Scene-Level Diffusioncited by 111
  28. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  29. 2024MagicDrive-V2: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Controlcited by 92
  30. 2023Language Conditioned Traffic Generationcited by 87
  31. 2024Manipulate-Anything: Automating Real-World Robots using Vision-Language Modelscited by 83
  32. 2024World Models for Autonomous Driving: An Initial Surveycited by 80
  33. 2019From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Followingcited by 67
  34. 2024Think2Drive: Efficient Reinforcement Learning by Thinking in Latent World Model for Quasi-Realistic Autonomous Driving (in CARLA-v2)cited by 52
  35. 2024Doe-1: Closed-Loop Autonomous Driving with Large World Modelcited by 51
  36. 2022Enhance Sample Efficiency and Robustness of End-to-end Urban Autonomous Driving via Semantic Masked World Modelcited by 45
  37. 2024Solving Motion Planning Tasks with a Scalable Generative Modelcited by 40
  38. 2024DreamForge: Motion-Aware Autoregressive Video Generation for Multi-View Driving Scenescited by 35

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Safety in Embodied AI: A Survey of Risks, Attacks, and Defensescited by 7
  2. 2026HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Taskcited by 5
  3. 2026OpenWorldLib: A Unified Codebase and Definition of Advanced World Modelscited by 3
  4. 2026Security of World-Model-Based Embodied AI: A Lifecycle of Threats, Defenses, and Evaluationcited by 2
  5. 2026From Digital Twins to World Models:Opportunities, Challenges, and Applications for Mobile Edge General Intelligencecited by 0
  6. 2026Security and Privacy in Large-Model-Driven Embodied Agents: Attacks, Defenses, and Future Directionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Comprehensive Survey on World Models for Embodied AI
  2. 2026Rethinking World Models for Safety-Critical Embodied Systems
  3. 2026World Model for Robot Learning: A Comprehensive Survey
  4. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Models
  5. 2025A Step Toward World Models: A Survey on Robotic Manipulation
  6. 2026Safety, Security, and Cognitive Risks in World Models
  7. 2026World Action Models: The Next Frontier in Embodied AI
  8. 2024HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environments