mlsanityData to 7 Oct 2026

Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Driving

arXiv 2603.27287 ↗First posted 28 Mar 2026cs.RO cs.CVcited by 9

On the map: World models → World models for autonomous driving

also in 1 more
Abstract (arXiv)

Autonomous driving requires reasoning about how the environment evolves and planning actions accordingly. Existing world-model-based approaches typically predict future scenes first and plan afterwards, resulting in open-loop imagination that may drift from the actual decision process. In this paper, we present Uni-World VLA, a unified vision-language-action (VLA) model that tightly interleaves future frame prediction and trajectory planning. Instead of generating a full world rollout before planning, our model alternates between predicting future frames and ego actions step by step, allowing planning decisions to be continuously conditioned on the imagined future observations. This interleaved generation forms a closed-loop interaction between world modeling and control, enabling more adaptive decision-making in dynamic traffic scenarios. In addition, we incorporate monocular depth information into frames to provide stronger geometric cues for world modeling, improving long-horizon scene prediction. Experiments on the NAVSIM benchmark show that our approach achieves competitive closed-loop planning performance while producing high-fidelity future frame predictions. These results demonstrate that tightly coupling world prediction and planning is a promising direction for scalable VLA driving systems.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 53 of its 62 references that are in the map. First posted on arXiv 28 Mar 2026.

Builds on

53 of 62 references.

Closest ancestors · 12

Same direction, newest first

  1. 2026ResWorld: Temporal Residual World Model for End-to-End Autonomous Driving11 Feb
  2. 2026UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving7 Jan
  3. 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving14 Oct
  4. 2025Epona: Autoregressive Diffusion World Model for Autonomous Driving30 Jun
  5. 2025GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action Control28 May
  6. 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Model2 Apr
  7. 2025VaViM and VaVAM: Autonomous Driving through Video Generative Modeling21 Feb
  8. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers24 Dec
  9. 2024Doe-1: Closed-Loop Autonomous Driving with Large World Model12 Dec
  10. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
  11. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving18 Sep
  12. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles22 Jun

Other references · 41

  1. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  3. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  4. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  5. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  6. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  7. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  8. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  9. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  10. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  11. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  12. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
29 more
  1. 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 401
  2. 2023GPT-Driver: Learning to Drive with GPTcited by 344
  3. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
  4. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
  5. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 287
  6. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
  7. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  8. 2024Global Structure-from-Motion Revisitedcited by 215
  9. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
  10. 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Drivingcited by 156
  11. 2025Pseudo-Simulation for Autonomous Drivingcited by 141
  12. 2024VLP: Vision Language Planning for Autonomous Drivingcited by 125
  13. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
  14. 2018CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-drivingcited by 96
  15. 2023WoVoGen: World Volume-aware Diffusion for Controllable Multi-camera Driving Scene Generationcited by 93
  16. 2025RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learningcited by 76
  17. 2025A Comprehensive Survey on World Models for Embodied AIcited by 65
  18. 2025DiffVLA: Vision-Language Guided Diffusion Planning for Autonomous Drivingcited by 64
  19. 2026Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memorycited by 37
  20. 2026SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Drivingcited by 34
  21. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
  22. 2024GaussianAD: Gaussian-Centric End-to-End Autonomous Drivingcited by 31
  23. 2025From Forecasting to Planning: Policy World Model for Collaborative State-Action Predictioncited by 29
  24. 2025What Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation Modelscited by 29
  25. 2025ImagiDrive: A Unified Imagination-and-Planning Framework for Autonomous Drivingcited by 24
  26. 2025Future-Aware End-to-End Driving: Bidirectional Modeling of Trajectory Planning and Scene Evolutioncited by 22
  27. 2024PreSight: Enhancing Autonomous Vehicle Perception with City-Scale NeRF Priorscited by 21
  28. 2025Neural Radiance Fields for the Real World: A Surveycited by 12
  29. 2025Perception in Plan: Coupled Perception and Planning for End-to-End Autonomous Drivingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning Vision-Language-Action World Models for Autonomous Driving
  2. 2026DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Driving
  3. 2026WorldVLM: Combining World Model Forecasting and Vision-Language Reasoning
  4. 2026RAF-VLA: Representation Alignment with the Future for End-to-End Autonomous Driving
  5. 2026WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving
  6. 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation
  7. 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving
  8. 2026HyWorldVLA: A Vision-Language-Action Model with Hybrid World Modeling for Autonomous Driving