mlsanityData to 14 Oct 2026

World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model

arXiv 2507.00603 ↗First posted 1 Jul 2025cs.CVcited by 95

On the map: World models → World models for autonomous driving

also in 1 more
Abstract (arXiv)

End-to-end autonomous driving directly generates planning trajectories from raw sensor data, yet it typically relies on costly perception supervision to extract scene information. A critical research challenge arises: constructing an informative driving world model to enable perception annotation-free, end-to-end planning via self-supervised learning. In this paper, we present World4Drive, an end-to-end autonomous driving framework that employs vision foundation models to build latent world models for generating and evaluating multi-modal planning trajectories. Specifically, World4Drive first extracts scene features, including driving intention and world latent representations enriched with spatial-semantic priors provided by vision foundation models. It then generates multi-modal planning trajectories based on current scene features and driving intentions and predicts multiple intention-driven future states within the latent space. Finally, it introduces a world model selector module to evaluate and select the best trajectory. We achieve perception annotation-free, end-to-end planning through self-supervised alignment between actual future observations and predicted observations reconstructed from the latent space. World4Drive achieves state-of-the-art performance without manual perception annotations on both the open-loop nuScenes and closed-loop NavSim benchmarks, demonstrating an 18.1% relative reduction in L2 error, 46.7% lower collision rate, and 3.75 faster training convergence. Codes will be accessed at https://github.com/ucaszyp/World4Drive.

Key facts. Cited by 95 papers in the map (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 47 of its 53 references that are in the map. First posted on arXiv 1 Jul 2025.

Builds on

47 of 53 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving11 Feb
  2. 2025Dream to Drive with Predictive Individual World Model28 Jan
  3. 2024DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Model14 Oct
  4. 2024Tokenize the World into Object-level Knowledge to Address Long-tail Events in Autonomous Driving1 Jul
  5. 2024Enhancing End-to-End Autonomous Driving with Latent World Model12 Jun
  6. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
  7. 2024DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Driving7 May
  8. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation11 Mar
  9. 2024GenAD: Generative End-to-End Autonomous Driving18 Feb
  10. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
  11. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving27 Nov
  12. 2023Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion2 Nov
1 more
  1. 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep

Other references · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  5. 2022Planning-oriented Autonomous Drivingcited by 1,087
  6. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  7. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  8. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  9. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  10. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
  11. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 395
  12. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 377
22 more
  1. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 368
  2. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 365
  3. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
  4. 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 314
  5. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  6. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 289
  7. 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 249
  8. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
  9. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 241
  10. 2023Scene as Occupancycited by 209
  11. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 162
  12. 2023LingoQA: Visual Question Answering for Autonomous Drivingcited by 145
  13. 2024VLP: Vision Language Planning for Autonomous Drivingcited by 126
  14. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 126
  15. 2025GoalFlow: Goal-Driven Flow Matching for Multimodal Trajectories Generation in End-to-End Autonomous Drivingcited by 120
  16. 2023ADAPT: Action-aware Driving Caption Transformercited by 78
  17. 2024MonoOcc: Digging into Monocular Semantic Occupancy Predictioncited by 44
  18. 2024TOD3Cap: Towards 3D Dense Captioning in Outdoor Scenescited by 40
  19. 2024PlanAgent: A Multi-modal Large Language Agent for Closed-loop Vehicle Motion Planningcited by 38
  20. 2023PPAD: Iterative Interactions of Prediction and Planning for End-to-end Autonomous Drivingcited by 33
  21. 2025UncAD: Towards Safe End-to-end Autonomous Driving via Online Map Uncertaintycited by 14
  22. 2024Data Scaling Laws for Imitation Learning-Based End-to-End Autonomous Drivingcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 20; 2026: 75.

95 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey on World Models for Embodied AIcited by 68
  2. 20253D and 4D World Modeling: A Surveycited by 67
  3. 2026Driving on Registerscited by 44
  4. 2026DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Drivingcited by 37
  5. 2025OmniNWM: Omniscient Driving Navigation World Modelscited by 33
  6. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 33
  7. 2026Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Drivingcited by 33
  8. 2026Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Drivingcited by 30
  9. 2025ImagiDrive: A Unified Imagination-and-Planning Framework for Autonomous Drivingcited by 25
  10. 2026DriveVA: Video Action Models are Zero-Shot Driverscited by 23
  11. 2026MultiWorld: Scalable Multi-Agent Multi-View Video World Modelscited by 23
  12. 2026UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Drivingcited by 20
38 more
  1. 2025MobileWorldBench: Towards Semantic World Modeling For Mobile Agentscited by 18
  2. 2026SpanVLA: Efficient Action Bridging and Learning from Negative-Recovery Samples for Vision-Language-Action Modelcited by 17
  3. 2026DriveFuture: Future-Aware Latent World Models for Autonomous Drivingcited by 17
  4. 2026ResWorld: Temporal Residual World Model for End-to-End Autonomous Drivingcited by 15
  5. 2026DreamerAD: Efficient Reinforcement Learning via Latent World Model for Autonomous Drivingcited by 15
  6. 2026EponaV2: Driving World Model with Comprehensive Future Reasoningcited by 15
  7. 2025AD-R1: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving with Impartial World Modelscited by 14
  8. 2026VGGT-World: Transforming VGGT into an Autoregressive Geometry World Modelcited by 13
  9. 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representationcited by 13
  10. 2025MindDrive: An All-in-One Framework Bridging World Models and Vision-Language Model for End-to-End Autonomous Drivingcited by 12
  11. 2026ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Drivingcited by 12
  12. 2026CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Drivingcited by 12
  13. 2025GS-Occ3D: Scaling Vision-only Occupancy Reconstruction with Gaussian Splattingcited by 11
  14. 2025GenieDrive: Towards Physics-Aware Driving World Model with 4D Occupancy Guided Video Generationcited by 11
  15. 2026Learning Vision-Language-Action World Models for Autonomous Drivingcited by 11
  16. 2026CLOVER: Closed-Loop Value Estimation and Ranking for End-to-End Autonomous Driving Planningcited by 11
  17. 2026TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulationcited by 11
  18. 2026Metis: A Generalizable and Efficient World-Action Model for Autonomous Driving and Urban Navigationcited by 11
  19. 2025TrajDiff: End-to-end Autonomous Driving without Perception Annotationcited by 8
  20. 2025TakeAD: Preference-based Post-optimization for End-to-end Autonomous Driving with Expert Takeover Datacited by 8
  21. 2026Learning from Mistakes: Post-Training for Driving VLA with Takeover Datacited by 8
  22. 2025FutureX: Enhance End-to-End Autonomous Driving via Latent Chain-of-Thought World Modelcited by 7
  23. 2026Test-Time Trajectory Optimization for Autonomous Drivingcited by 7
  24. 2026Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rolloutcited by 6
  25. 2026GraphWorld: Long-Horizon Planning with World Models for End-to-End Autonomous Drivingcited by 6
  26. 2026World-Action Models for Robot Learning and Control: A Surveycited by 6
  27. 2025Enhancing Physical Consistency in Lightweight World Modelscited by 5
  28. 2026Self-Supervised JEPA-based World Models for LiDAR Occupancy Completion and Forecastingcited by 5
  29. 2026DA-WAM: Decision-Aligned Future Latents for Driving World Modelscited by 5
  30. 2025WPT: World-to-Policy Transfer via Online World Model Distillationcited by 4
  31. 2025Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systemscited by 4
  32. 2026Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Drivingcited by 4
  33. 2025Mimir: Hierarchical Goal-Driven Diffusion with Uncertainty Propagation for End-to-End Autonomous Drivingcited by 3
  34. 2026UniDWM: Towards a Unified Driving World Model via Multifaceted Representation Learningcited by 3
  35. 2026Latent World Models for Automated Driving: A Unified Taxonomy, Evaluation Framework, and Open Challengescited by 3
  36. 2026Zero-Shot Cross-City Generalization in End-to-End Autonomous Driving: Self-Supervised versus Supervised Representationscited by 3
  37. 2026Bench2Drive-Robust: Benchmarking Closed-Loop Autonomous Driving under Deployment Perturbationscited by 3
  38. 2026Distill to Think, Foresee to Act: Cognitive-Physical Reinforcement Learning for Autonomous Drivingcited by 3

50 most cited of 95 All 95 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025WorldRFT: Latent World Model Planning with Reinforcement Fine-Tuning for Autonomous Driving
  2. 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving
  3. 2026PlanWAM: Planning-Shaped Future Representations for End-to-End Autonomous Driving
  4. 2026ForeDrive: Foresight-Guided End-to-End Autonomous Driving with a Planning-Relevant Latent World Model
  5. 2025Future-Aware End-to-End Driving: Bidirectional Modeling of Trajectory Planning and Scene Evolution
  6. 2026ReDrive: Shaping Representations with World Modeling for End-to-End Driving
  7. 2026Planning-Oriented End-to-End Autonomous Driving: Architectures, Evaluation, and Emerging Paradigms
  8. 2026SparseWorld: Enhancing End-to-End Autonomous Driving via World Models with Sparse Scene Representation