UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving
On the map: World models → World models for autonomous driving
also in 1 more
Abstract (arXiv)
World models have become central to autonomous driving, where accurate scene understanding and future prediction are crucial for safe control. Recent work has explored using vision-language models (VLMs) for planning, yet existing approaches typically treat perception, prediction, and planning as separate modules. We propose UniDrive-WM, a unified VLM-based world model that jointly performs driving-scene understanding, trajectory planning, and trajectory-conditioned future image generation within a single architecture. UniDrive-WM's trajectory planner predicts a future trajectory, which conditions a VLM-based image generator to produce plausible future frames. These predictions provide additional supervisory signals that enhance scene understanding and iteratively refine trajectory generation. We further compare discrete and continuous output representations for future image prediction, analyzing their influence on downstream driving performance. Experiments on the challenging Bench2Drive benchmark show that UniDrive-WM produces high-fidelity future images and improves planning performance by 7.3% in L2 trajectory error and 10.4% in collision rate over the previous best method. These results demonstrate the advantages of tightly integrating VLM-driven reasoning, planning, and generative world modeling for autonomous driving. The project page is available at https://unidrive-wm.github.io/UniDrive-WM.
Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 59 of its 67 references that are in the map. First posted on arXiv 7 Jan 2026.
Builds on
59 of 67 references.
Closest ancestors · 15
Same direction, newest first
- 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving14 Oct
- 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model1 Jul
- 2025Epona: Autoregressive Diffusion World Model for Autonomous Driving30 Jun
- 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning6 Apr
- 2024Doe-1: Closed-Loop Autonomous Driving with Large World Model12 Dec
- 2024OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Driving5 Sep
- 2024BEVWorld: A Multimodal World Simulator for Autonomous Driving via Scene-Level BEV Latents8 Jul
- 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
- 2024GenAD: Generative End-to-End Autonomous Driving18 Feb
- 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
- 2023Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion2 Nov
- 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep
3 more
Other references · 44
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2022Planning-oriented Autonomous Drivingcited by 1,087
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
- 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
- 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
- 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
32 more
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
- 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 451
- 2023EVA-02: A Visual Representation for Neon Genesiscited by 381
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
- 2025WorldVLA: Towards Autoregressive Action World Modelcited by 367
- 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 252
- 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 248
- 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 206
- 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 206
- 2025Unified Video Action Modelcited by 205
- 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenescited by 191
- 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 178
- 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 169
- 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 166
- 2023DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Drivingcited by 144
- 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 126
- 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
- 2025Don't Shake the Wheel: Momentum-Aware Planning in End-to-End Autonomous Drivingcited by 84
- 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 78
- 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 78
- 2025Thinking with Generated Imagescited by 73
- 2024LidarDM: Generative LiDAR Simulation in a Generated Worldcited by 54
- 2025BLIP3o-NEXT: Next Frontier of Native Image Generationcited by 54
- 2025Pre-Trained Video Generative Models as World Simulatorscited by 44
- 2024Language-Image Models with 3D Understandingcited by 31
- 2024HoloDrive: Holistic 2D-3D Multi-Modal Street Scene Generation for Autonomous Drivingcited by 23
- 2024SimpleLLM4AD: An End-to-End Vision-Language Model with Graph Visual Question Answering for Autonomous Drivingcited by 16
- 2024Extrapolated Urban View Synthesis Benchmarkcited by 13
- 2025Towards foundational LiDAR world models with efficient latent flow matchingcited by 10
- 2025Tracking Meets Large Multimodal Models for Driving Scenario Understandingcited by 7
- 2025Work Zones challenge VLM Trajectory Planning: Toward Mitigation and Robust Autonomous Drivingcited by 4
- 2025LiDARDraft: Generating LiDAR Point Cloud from Versatile Inputscited by 1
Built on by
Citing papers per year: 2026: 20.
20 papers.
Sort:
By area:
- 2026World Model for Robot Learning: A Comprehensive Surveycited by 46
- 2026DriveVA: Video Action Models are Zero-Shot Driverscited by 23
- 2026ActWorld: From Explorable to Interactive World Model via Action-Aware Memorycited by 14
- 2026ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Drivingcited by 12
- 2026Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Drivingcited by 11
- 2026Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rolloutcited by 6
- 2026Self-Improving World Modelling with Latent Actionscited by 5
- 2026A Definition and Roadmap for World Modelscited by 5
- 2026Pondering the Way: Spatial-perceiving World Action Model for Embodied Navigationcited by 3
- 2026GeoWAM: Visual Geometry World Action Models for Autonomous Drivingcited by 3
- 2026WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Drivingcited by 1
- 2026World Models for Embodied Intelligence: From Plausible to Controllable to Actionablecited by 1
8 more
- 2026Orion-Lite: Distilling LLM Reasoning into Efficient Vision-Only Driving Modelscited by 0
- 2026LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Modelcited by 0
- 2026WildCity: A Real-World City-Scale Testbed for Rendering, Simulation, and Spatial Intelligencecited by 0
- 2026Latent-Centroid Steering: Single-Pass Classifier-Free Guidance for Command-Aligned Autonomous Drivingcited by 0
- 2026DF3: World Modeling via Decoder-Free Feature Forecasting in Autonomous Navigationcited by 0
- 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?cited by 0
- 2026Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Drivingcited by 0
- 2026Valerant: An Automatic Navigable Game Map Generator via Action-Conditioned World Model Explorationcited by 0
- 2026World Models for Embodied Intelligence: From Plausible to Controllable to Actionable15 Sep 2026
- 2026Valerant: An Automatic Navigable Game Map Generator via Action-Conditioned World Model Exploration8 Sep 2026
- 2026Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Driving31 Aug 2026
- 2026GeoWAM: Visual Geometry World Action Models for Autonomous Driving24 Aug 2026
- 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?24 Aug 2026
- 2026DF3: World Modeling via Decoder-Free Feature Forecasting in Autonomous Navigation3 Aug 2026
- 2026Latent-Centroid Steering: Single-Pass Classifier-Free Guidance for Command-Aligned Autonomous Driving31 Jul 2026
- 2026WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving9 Jul 2026
- 2026WildCity: A Real-World City-Scale Testbed for Rendering, Simulation, and Spatial Intelligence7 Jul 2026
- 2026A Definition and Roadmap for World Models7 Jul 2026
- 2026Pondering the Way: Spatial-perceiving World Action Model for Embodied Navigation29 Jun 2026
- 2026ActWorld: From Explorable to Interactive World Model via Action-Aware Memory16 Jun 2026
8 more
- 2026LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Model21 May 2026
- 2026Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rollout6 May 2026
- 2026World Model for Robot Learning: A Comprehensive Survey30 Apr 2026
- 2026Orion-Lite: Distilling LLM Reasoning into Efficient Vision-Only Driving Models9 Apr 2026
- 2026DriveVA: Video Action Models are Zero-Shot Drivers5 Apr 2026
- 2026ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Driving3 Apr 2026
- 2026Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Driving28 Mar 2026
- 2026Self-Improving World Modelling with Latent Actions5 Feb 2026
- 2026World Model for Robot Learning: A Comprehensive Surveycited by 46
- 2026DriveVA: Video Action Models are Zero-Shot Driverscited by 23
- 2026ActWorld: From Explorable to Interactive World Model via Action-Aware Memorycited by 14
- 2026ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Drivingcited by 12
- 2026Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Drivingcited by 11
- 2026Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rolloutcited by 6
- 2026Self-Improving World Modelling with Latent Actionscited by 5
- 2026A Definition and Roadmap for World Modelscited by 5
- 2026Pondering the Way: Spatial-perceiving World Action Model for Embodied Navigationcited by 3
- 2026GeoWAM: Visual Geometry World Action Models for Autonomous Drivingcited by 3
- 2026World Models for Embodied Intelligence: From Plausible to Controllable to Actionablecited by 1
- 2026LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Modelcited by 0
12 most cited of 15 All 15 (JSON)
- 2026World Models for Embodied Intelligence: From Plausible to Controllable to Actionable15 Sep 2026
- 2026Valerant: An Automatic Navigable Game Map Generator via Action-Conditioned World Model Exploration8 Sep 2026
- 2026GeoWAM: Visual Geometry World Action Models for Autonomous Driving24 Aug 2026
- 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?24 Aug 2026
- 2026DF3: World Modeling via Decoder-Free Feature Forecasting in Autonomous Navigation3 Aug 2026
- 2026A Definition and Roadmap for World Models7 Jul 2026
- 2026Pondering the Way: Spatial-perceiving World Action Model for Embodied Navigation29 Jun 2026
- 2026ActWorld: From Explorable to Interactive World Model via Action-Aware Memory16 Jun 2026
- 2026LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Model21 May 2026
- 2026Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rollout6 May 2026
- 2026World Model for Robot Learning: A Comprehensive Survey30 Apr 2026
- 2026DriveVA: Video Action Models are Zero-Shot Drivers5 Apr 2026
12 newest of 15 All 15 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation
- 2026UniDWM: Towards a Unified Driving World Model via Multifaceted Representation Learning
- 2026WorldVLM: Combining World Model Forecasting and Vision-Language Reasoning
- 2026LWDrive: Layer-Wise World-Model-Guided Vision-Language Model Planning for Autonomous Driving
- 2025UniUGP: Unifying Understanding, Generation, and Planing For End-to-end Autonomous Driving
- 2026Learning Vision-Language-Action World Models for Autonomous Driving
- 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving
- 2025Percept-WAM: Perception-Enhanced World-Awareness-Action Model for Robust End-to-End Autonomous Driving