mlsanityData to 14 Oct 2026

The Role of World Models in Shaping Autonomous Driving: A Comprehensive Survey

arXiv 2502.10498 ↗First posted 14 Feb 2025cs.CVcited by 29

On the map: World models → World models for autonomous driving

also in 1 more
Abstract (arXiv)

The Driving World Model (DWM), which focuses on predicting scene evolution during the driving process, has emerged as a promising paradigm in the pursuit of autonomous driving (AD). DWMs enable AD systems to better perceive, understand, and interact with dynamic driving environments. In this survey, we provide a comprehensive overview of the latest progress in DWM. First, we review the DWM ecosystem, which is constructed using mainstream simulators, high-impact datasets, and various metrics that evaluate DWMs across multiple dimensions. We then categorize existing approaches based on the modalities of the predicted scenes, including video, point cloud, occupancy, latent feature, and traffic map, and summarize their specific applications in AD research. In addition, the performance of representative approaches across generating and driving tasks is presented. Finally, we discuss the potential limitations of current research and propose future directions. This survey provides valuable insights into the development and application of DWM, fostering its broader adoption in AD. The relevant papers are collected at https://github.com/LMD0311/Awesome-World-Model.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 102 of its 176 references that are in the map. First posted on arXiv 14 Feb 2025.

Builds on

102 of 176 references.

Closest ancestors · 49

Same direction, newest first

  1. 2025Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving11 Feb
  2. 2025Dream to Drive with Predictive Individual World Model28 Jan
  3. 2025HERMES: A Unified Self-Driving World Model for Simultaneous 3D Scene Understanding and Generation24 Jan
  4. 2025AdaWM: Adaptive World Model based Planning for Autonomous Driving22 Jan
  5. 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT27 Dec
  6. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers24 Dec
  7. 2024An Efficient Occupancy World Model via Decoupled Dynamic Flow and Image-assisted Training18 Dec
  8. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Prediction13 Dec
  9. 2024Doe-1: Closed-Loop Autonomous Driving with Large World Model12 Dec
  10. 2024Physical Informed Driving World Model11 Dec
  11. 2024ACT-Bench: Towards Action Controllable World Models for Autonomous Driving6 Dec
  12. 2024InfiniCube: Unbounded and Controllable Dynamic 3D Driving Scene Generation with World-Guided Video Models5 Dec
37 more
  1. 2024ReconDreamer: Crafting World Models for Driving Scene Reconstruction via Online Restoration29 Nov
  2. 2024Exploring the Interplay Between Video Generation and World Models in Autonomous Driving: A Survey5 Nov
  3. 2024DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenes23 Oct
  4. 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation17 Oct
  5. 2024DrivingDojo Dataset: Advancing Interactive and Knowledge-Enriched Driving World Model14 Oct
  6. 2024DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Model14 Oct
  7. 2024Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Models25 Sep
  8. 2024Learning Multiple Probabilistic Decisions from Latent World Model in Autonomous Driving24 Sep
  9. 2024RenderWorld: World Model with Self-Supervised 3D Label17 Sep
  10. 2024OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Driving5 Sep
  11. 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving26 Aug
  12. 2024Planning with Adaptive World Models for Autonomous Driving15 Jun
  13. 2024UnO: Unsupervised Occupancy Fields for Perception and Forecasting12 Jun
  14. 2024Enhancing End-to-End Autonomous Driving with Latent World Model12 Jun
  15. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
  16. 2024MagicDrive3D: Controllable 3D Generation for Any-View Rendering in Street Scenes23 May
  17. 2024DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Driving7 May
  18. 2024GenAD: Generalized Predictive Model for Autonomous Driving14 Mar
  19. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation11 Mar
  20. 2024World Models for Autonomous Driving: An Initial Survey5 Mar
  21. 2024Think2Drive: Efficient Reinforcement Learning by Thinking in Latent World Model for Quasi-Realistic Autonomous Driving (in CARLA-v2)26 Feb
  22. 2023Visual Point Cloud Forecasting enables Scalable Autonomous Driving29 Dec
  23. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
  24. 2023Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications29 Nov
  25. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving27 Nov
  26. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representations20 Nov
  27. 2023Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion2 Nov
  28. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research12 Oct
  29. 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep
  30. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving18 Sep
  31. 2023UniWorld: Autonomous Driving Pre-training via World Models14 Aug
  32. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving27 Apr
  33. 2023TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction7 Mar
  34. 2022Enhance Sample Efficiency and Robustness of End-to-end Urban Autonomous Driving via Semantic Masked World Model8 Oct
  35. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning26 Sep
  36. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles22 Jun
  37. 2017CARLA: An Open Urban Driving Simulator10 Nov

Other references · 53

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  4. 2022Scalable Diffusion Models with Transformerscited by 6,054
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  6. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  7. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  8. 2018World Modelscited by 1,513
  9. 2022Planning-oriented Autonomous Drivingcited by 1,087
  10. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  11. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  12. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
38 more
  1. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  2. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
  3. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 422
  4. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
  5. 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 328
  6. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 252
  7. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 252
  8. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 221
  9. 2023Scene as Occupancycited by 209
  10. 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 168
  11. 2024PointMamba: A Simple State Space Model for Point Cloud Analysiscited by 167
  12. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
  13. 2022Model-Based Imitation Learning for Urban Drivingcited by 150
  14. 2022SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Modelscited by 147
  15. 2023Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Drivingcited by 123
  16. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  17. 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 106
  18. 2024DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Drivingcited by 98
  19. 2023WoVoGen: World Volume-aware Diffusion for Controllable Multi-camera Driving Scene Generationcited by 93
  20. 2024MagicDrive-V2: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Controlcited by 92
  21. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 78
  22. 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 78
  23. 2024LidarDM: Generative LiDAR Simulation in a Generated Worldcited by 54
  24. 2022Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Modelscited by 53
  25. 2024Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysiscited by 53
  26. 2024SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rolloutcited by 48
  27. 2024RangeLDM: Fast Realistic LiDAR Point Cloud Generationcited by 47
  28. 2024DINO-Foresight: Looking into the Future with DINOcited by 45
  29. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Drivingcited by 44
  30. 2024Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learningcited by 43
  31. 2024SimGen: Simulator-conditioned Driving Scene Generationcited by 34
  32. 2024DrivingSphere: Building a High-fidelity 4D World for Closed-loop Simulationcited by 34
  33. 2025Occ-LLM: Enhancing Autonomous Driving with Occupancy-Based Large Language Modelscited by 28
  34. 2024UniMLVG: Unified Framework for Multi-view Long Video Generation with Comprehensive Control Capabilities for Autonomous Drivingcited by 27
  35. 2025Black-Box Adversarial Attack on Vision Language Models for Autonomous Drivingcited by 24
  36. 2024HoloDrive: Holistic 2D-3D Multi-Modal Street Scene Generation for Autonomous Drivingcited by 23
  37. 2024Causal Composition Diffusion Model for Closed-loop Traffic Generationcited by 22
  38. 2023LimSim: A Long-term Interactive Multi-scenario Traffic Simulatorcited by 13

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 18.

29 papers.

Sort:

By area:

  1. 2025RELIC: Interactive Video World Model with Long-Horizon Memorycited by 96
  2. 20253D and 4D World Modeling: A Surveycited by 67
  3. 2025Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysiscited by 38
  4. 2026DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Drivingcited by 37
  5. 2025GaussianDWM: 3D Gaussian Driving World Model for Unified Scene Understanding and Multi-Modal Generationcited by 23
  6. 2025UniFuture: A 4D Driving World Model for Future Generation and Perceptioncited by 22
  7. 2025Mind with Eyes: from Language Reasoning to Multimodal Reasoningcited by 22
  8. 2026DriveFuture: Future-Aware Latent World Models for Autonomous Drivingcited by 17
  9. 2025DriVerse: Navigation World Model for Driving Simulation via Multimodal Trajectory Prompting and Motion Alignmentcited by 16
  10. 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 14
  11. 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representationcited by 13
  12. 2025Chain-of-Thought for Autonomous Driving: A Comprehensive Survey and Future Prospectscited by 12
17 more
  1. 2025UniPR-3D: Towards Universal Visual Place Recognition with Visual Geometry Grounded Transformercited by 8
  2. 2025U-ViLAR: Uncertainty-Aware Visual Localization for Autonomous Driving via Differentiable Association and Registrationcited by 5
  3. 2025From 2D to 3D Cognition: A Brief Survey of General World Modelscited by 4
  4. 2026GeoStream: Toward Precise Camera Controlled Streaming Video Generationcited by 4
  5. 2026Research on World Models Is Not Merely Injecting World Knowledge into Specific Taskscited by 3
  6. 2026OpenWorldLib: A Unified Codebase and Definition of Advanced World Modelscited by 3
  7. 2026Annotation-Free Detection of Drivable Areas and Curbs Leveraging LiDAR Point Cloud Mapscited by 2
  8. 2026Representations Before Pixels: Semantics-Guided Hierarchical Video Predictioncited by 1
  9. 2026Infrastructure-Centric World Models: Bridging Temporal Depth and Spatial Breadth for Roadside Perceptioncited by 1
  10. 2026HyWorldVLA: A Vision-Language-Action Model with Hybrid World Modeling for Autonomous Drivingcited by 1
  11. 2026TPS-Drive: Task-Guided Representation Purification for VLM-based Autonomous Drivingcited by 0
  12. 2026Quo Vadis, World Modeling?cited by 0
  13. 2026GaussianDWM++: Language-Grounded 3D Gaussian Driving World Model for Unified Scene Understanding, Editing, and Multi-Modal Generationcited by 0
  14. 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?cited by 0
  15. 2026How Far Can 5,500 Hours of Driving Take You? A Scaling Law Analysis of Video Diffusion Modelscited by 0
  16. 2026WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Drivingcited by 0
  17. 2026A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platformcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey of World Models for Autonomous Driving
  2. 2026From Virtual Environments to Real-World Trials: Emerging Trends in Autonomous Driving
  3. 2025ProphetDWM: A Driving World Model for Rolling Out Future Actions and Videos
  4. 2024InfinityDrive: Breaking Time Limits in Driving World Models
  5. 2026DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving
  6. 2026Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving
  7. 2026VehDyn: A Driving World Model Benchmark for Vehicle Dynamics
  8. 2023Vision Language Models in Autonomous Driving: A Survey and Outlook