Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Driving
On the map: World models → World models for autonomous driving
also in 1 more
Abstract (arXiv)
Autonomous driving requires reasoning about how the environment evolves and planning actions accordingly. Existing world-model-based approaches typically predict future scenes first and plan afterwards, resulting in open-loop imagination that may drift from the actual decision process. In this paper, we present Uni-World VLA, a unified vision-language-action (VLA) model that tightly interleaves future frame prediction and trajectory planning. Instead of generating a full world rollout before planning, our model alternates between predicting future frames and ego actions step by step, allowing planning decisions to be continuously conditioned on the imagined future observations. This interleaved generation forms a closed-loop interaction between world modeling and control, enabling more adaptive decision-making in dynamic traffic scenarios. In addition, we incorporate monocular depth information into frames to provide stronger geometric cues for world modeling, improving long-horizon scene prediction. Experiments on the NAVSIM benchmark show that our approach achieves competitive closed-loop planning performance while producing high-fidelity future frame predictions. These results demonstrate that tightly coupling world prediction and planning is a promising direction for scalable VLA driving systems.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 53 of its 62 references that are in the map. First posted on arXiv 28 Mar 2026.
Builds on
53 of 62 references.
Closest ancestors · 12
Same direction, newest first
- 2026ResWorld: Temporal Residual World Model for End-to-End Autonomous Driving11 Feb
- 2026UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving7 Jan
- 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving14 Oct
- 2025Epona: Autoregressive Diffusion World Model for Autonomous Driving30 Jun
- 2025GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action Control28 May
- 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Model2 Apr
- 2025VaViM and VaVAM: Autonomous Driving through Video Generative Modeling21 Feb
- 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers24 Dec
- 2024Doe-1: Closed-Loop Autonomous Driving with Large World Model12 Dec
- 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
- 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving18 Sep
- 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles22 Jun
Other references · 41
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
- 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
- 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
- 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
- 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
- 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
29 more
- 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 401
- 2023GPT-Driver: Learning to Drive with GPTcited by 344
- 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
- 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
- 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 287
- 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
- 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
- 2024Global Structure-from-Motion Revisitedcited by 215
- 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
- 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Drivingcited by 156
- 2025Pseudo-Simulation for Autonomous Drivingcited by 141
- 2024VLP: Vision Language Planning for Autonomous Drivingcited by 125
- 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
- 2018CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-drivingcited by 96
- 2023WoVoGen: World Volume-aware Diffusion for Controllable Multi-camera Driving Scene Generationcited by 93
- 2025RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learningcited by 76
- 2025A Comprehensive Survey on World Models for Embodied AIcited by 65
- 2025DiffVLA: Vision-Language Guided Diffusion Planning for Autonomous Drivingcited by 64
- 2026Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memorycited by 37
- 2026SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Drivingcited by 34
- 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
- 2024GaussianAD: Gaussian-Centric End-to-End Autonomous Drivingcited by 31
- 2025From Forecasting to Planning: Policy World Model for Collaborative State-Action Predictioncited by 29
- 2025What Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation Modelscited by 29
- 2025ImagiDrive: A Unified Imagination-and-Planning Framework for Autonomous Drivingcited by 24
- 2025Future-Aware End-to-End Driving: Bidirectional Modeling of Trajectory Planning and Scene Evolutioncited by 22
- 2024PreSight: Enhancing Autonomous Vehicle Perception with City-Scale NeRF Priorscited by 21
- 2025Neural Radiance Fields for the Real World: A Surveycited by 12
- 2025Perception in Plan: Coupled Perception and Planning for End-to-End Autonomous Drivingcited by 3
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Drivingcited by 10
- 2026Metis: A Generalizable and Efficient World-Action Model for Autonomous Driving and Urban Navigationcited by 10
- 2026nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Drivingcited by 8
- 2026The DAWN of World-Action Interactive Modelscited by 7
- 2026SimWAM: A Simple World Action Model for End-to-End Autonomous Drivingcited by 5
- 2026ForgeDrive: Bidirectional Cross-Conditioning for Unified Visual-Action Generation in Autonomous Drivingcited by 2
- 2026Multi-Agent Embodied Autonomous Driving (MAEAD): From V2X Information Exchange to Shared World Modelscited by 0
- 2026SlackDrive: Reclaiming Runtime Slack for Adaptive Driving Inferencecited by 0
- 2026RoXDrive: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving via Action-Faithful Rolloutscited by 0
- 2026RoXDrive: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving via Action-Faithful Rollouts29 Sep 2026
- 2026SlackDrive: Reclaiming Runtime Slack for Adaptive Driving Inference23 Sep 2026
- 2026SimWAM: A Simple World Action Model for End-to-End Autonomous Driving7 Aug 2026
- 2026ForgeDrive: Bidirectional Cross-Conditioning for Unified Visual-Action Generation in Autonomous Driving30 Jun 2026
- 2026Metis: A Generalizable and Efficient World-Action Model for Autonomous Driving and Urban Navigation14 Jun 2026
- 2026Multi-Agent Embodied Autonomous Driving (MAEAD): From V2X Information Exchange to Shared World Models11 Jun 2026
- 2026nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Driving29 May 2026
- 2026The DAWN of World-Action Interactive Models12 May 2026
- 2026CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Driving11 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026Learning Vision-Language-Action World Models for Autonomous Driving
- 2026DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Driving
- 2026WorldVLM: Combining World Model Forecasting and Vision-Language Reasoning
- 2026RAF-VLA: Representation Alignment with the Future for End-to-End Autonomous Driving
- 2026WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving
- 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation
- 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving
- 2026HyWorldVLA: A Vision-Language-Action Model with Hybrid World Modeling for Autonomous Driving