mlsanityData to 14 Oct 2026

UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving

arXiv 2601.04453 ↗First posted 7 Jan 2026cs.CVcited by 20

On the map: World models → World models for autonomous driving

also in 1 more
Abstract (arXiv)

World models have become central to autonomous driving, where accurate scene understanding and future prediction are crucial for safe control. Recent work has explored using vision-language models (VLMs) for planning, yet existing approaches typically treat perception, prediction, and planning as separate modules. We propose UniDrive-WM, a unified VLM-based world model that jointly performs driving-scene understanding, trajectory planning, and trajectory-conditioned future image generation within a single architecture. UniDrive-WM's trajectory planner predicts a future trajectory, which conditions a VLM-based image generator to produce plausible future frames. These predictions provide additional supervisory signals that enhance scene understanding and iteratively refine trajectory generation. We further compare discrete and continuous output representations for future image prediction, analyzing their influence on downstream driving performance. Experiments on the challenging Bench2Drive benchmark show that UniDrive-WM produces high-fidelity future images and improves planning performance by 7.3% in L2 trajectory error and 10.4% in collision rate over the previous best method. These results demonstrate the advantages of tightly integrating VLM-driven reasoning, planning, and generative world modeling for autonomous driving. The project page is available at https://unidrive-wm.github.io/UniDrive-WM.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 59 of its 67 references that are in the map. First posted on arXiv 7 Jan 2026.

Builds on

59 of 67 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving14 Oct
  2. 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model1 Jul
  3. 2025Epona: Autoregressive Diffusion World Model for Autonomous Driving30 Jun
  4. 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning6 Apr
  5. 2024Doe-1: Closed-Loop Autonomous Driving with Large World Model12 Dec
  6. 2024OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Driving5 Sep
  7. 2024BEVWorld: A Multimodal World Simulator for Autonomous Driving via Scene-Level BEV Latents8 Jul
  8. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
  9. 2024GenAD: Generative End-to-End Autonomous Driving18 Feb
  10. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
  11. 2023Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion2 Nov
  12. 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep
3 more
  1. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving18 Sep
  2. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers31 Mar
  3. 2017CARLA: An Open Urban Driving Simulator10 Nov

Other references · 44

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  5. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  7. 2022Planning-oriented Autonomous Drivingcited by 1,087
  8. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  9. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  10. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  11. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  12. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
32 more
  1. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  2. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 451
  3. 2023EVA-02: A Visual Representation for Neon Genesiscited by 381
  4. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
  5. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 367
  6. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 252
  7. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 248
  8. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 206
  9. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 206
  10. 2025Unified Video Action Modelcited by 205
  11. 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenescited by 191
  12. 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 178
  13. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 169
  14. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 166
  15. 2023DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Drivingcited by 144
  16. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 126
  17. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  18. 2025Don't Shake the Wheel: Momentum-Aware Planning in End-to-End Autonomous Drivingcited by 84
  19. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 78
  20. 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 78
  21. 2025Thinking with Generated Imagescited by 73
  22. 2024LidarDM: Generative LiDAR Simulation in a Generated Worldcited by 54
  23. 2025BLIP3o-NEXT: Next Frontier of Native Image Generationcited by 54
  24. 2025Pre-Trained Video Generative Models as World Simulatorscited by 44
  25. 2024Language-Image Models with 3D Understandingcited by 31
  26. 2024HoloDrive: Holistic 2D-3D Multi-Modal Street Scene Generation for Autonomous Drivingcited by 23
  27. 2024SimpleLLM4AD: An End-to-End Vision-Language Model with Graph Visual Question Answering for Autonomous Drivingcited by 16
  28. 2024Extrapolated Urban View Synthesis Benchmarkcited by 13
  29. 2025Towards foundational LiDAR world models with efficient latent flow matchingcited by 10
  30. 2025Tracking Meets Large Multimodal Models for Driving Scenario Understandingcited by 7
  31. 2025Work Zones challenge VLM Trajectory Planning: Toward Mitigation and Robust Autonomous Drivingcited by 4
  32. 2025LiDARDraft: Generating LiDAR Point Cloud from Versatile Inputscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 20.

20 papers.

Sort:

By area:

  1. 2026World Model for Robot Learning: A Comprehensive Surveycited by 46
  2. 2026DriveVA: Video Action Models are Zero-Shot Driverscited by 23
  3. 2026ActWorld: From Explorable to Interactive World Model via Action-Aware Memorycited by 14
  4. 2026ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Drivingcited by 12
  5. 2026Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Drivingcited by 11
  6. 2026Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rolloutcited by 6
  7. 2026Self-Improving World Modelling with Latent Actionscited by 5
  8. 2026A Definition and Roadmap for World Modelscited by 5
  9. 2026Pondering the Way: Spatial-perceiving World Action Model for Embodied Navigationcited by 3
  10. 2026GeoWAM: Visual Geometry World Action Models for Autonomous Drivingcited by 3
  11. 2026WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Drivingcited by 1
  12. 2026World Models for Embodied Intelligence: From Plausible to Controllable to Actionablecited by 1
8 more
  1. 2026Orion-Lite: Distilling LLM Reasoning into Efficient Vision-Only Driving Modelscited by 0
  2. 2026LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Modelcited by 0
  3. 2026WildCity: A Real-World City-Scale Testbed for Rendering, Simulation, and Spatial Intelligencecited by 0
  4. 2026Latent-Centroid Steering: Single-Pass Classifier-Free Guidance for Command-Aligned Autonomous Drivingcited by 0
  5. 2026DF3: World Modeling via Decoder-Free Feature Forecasting in Autonomous Navigationcited by 0
  6. 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?cited by 0
  7. 2026Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Drivingcited by 0
  8. 2026Valerant: An Automatic Navigable Game Map Generator via Action-Conditioned World Model Explorationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation
  2. 2026UniDWM: Towards a Unified Driving World Model via Multifaceted Representation Learning
  3. 2026WorldVLM: Combining World Model Forecasting and Vision-Language Reasoning
  4. 2026LWDrive: Layer-Wise World-Model-Guided Vision-Language Model Planning for Autonomous Driving
  5. 2025UniUGP: Unifying Understanding, Generation, and Planing For End-to-end Autonomous Driving
  6. 2026Learning Vision-Language-Action World Models for Autonomous Driving
  7. 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving
  8. 2025Percept-WAM: Perception-Enhanced World-Awareness-Action Model for Robust End-to-End Autonomous Driving