mlsanityData to 7 Oct 2026

Learning Vision-Language-Action World Models for Autonomous Driving

arXiv 2604.09059 ↗First posted 10 Apr 2026cs.CV cs.AIcited by 11

On the map: World models → World models for autonomous driving

also in 2 more
Abstract (arXiv)

Vision-Language-Action (VLA) models have recently achieved notable progress in end-to-end autonomous driving by integrating perception, reasoning, and control within a unified multimodal framework. However, they often lack explicit modeling of temporal dynamics and global world consistency, which limits their foresight and safety. In contrast, world models can simulate plausible future scenes but generally struggle to reason about or evaluate the imagined future they generate. In this work, we present VLA-World, a simple yet effective VLA world model that unifies predictive imagination with reflective reasoning to improve driving foresight. VLA-World first uses an action-derived feasible trajectory to guide the generation of the next-frame image, capturing rich spatial and temporal cues that describe how the surrounding environment evolves. The model then reasons over this self-generated future imagined frame to refine the predicted trajectory, achieving higher performance and better interpretability. To support this pipeline, we curate nuScenes-GR-20K, a generative reasoning dataset derived from nuScenes, and employ a three-stage training strategy that includes pretraining, supervised fine-tuning, and reinforcement learning. Extensive experiments demonstrate that VLA-World consistently surpasses state-of-the-art VLA and world-model baselines on both planning and future-generation benchmarks. Project page: https://vlaworld.github.io

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 82 of its 87 references that are in the map. First posted on arXiv 10 Apr 2026.

Builds on

82 of 87 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025OmniNWM: Omniscient Driving Navigation World Models21 Oct
  2. 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model1 Jul
  3. 2025ReSim: Reliable World Simulation for Autonomous Driving11 Jun
  4. 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning6 Apr
  5. 2025Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving11 Feb
  6. 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT27 Dec
  7. 2024Doe-1: Closed-Loop Autonomous Driving with Large World Model12 Dec
  8. 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving26 Aug
  9. 2024DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Driving7 May
  10. 2024GenAD: Generalized Predictive Model for Autonomous Driving14 Mar
  11. 2024GenAD: Generative End-to-End Autonomous Driving18 Feb
  12. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
4 more
  1. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving27 Nov
  2. 2023ADriver-I: A General World Model for Autonomous Driving22 Nov
  3. 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep
  4. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving18 Sep

Other references · 66

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2025Qwen2.5-VL Technical Reportcited by 5,418
  9. 2017Neural Discrete Representation Learningcited by 4,650
  10. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  11. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  12. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
38 more
  1. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  5. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  6. 2024Qwen2 Technical Reportcited by 2,076
  7. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  8. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  9. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  10. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  11. 2022Planning-oriented Autonomous Drivingcited by 1,068
  12. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
  13. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  14. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  15. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
  16. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  17. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  18. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
  19. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  20. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
  21. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
  22. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
  23. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 287
  24. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
  25. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
  26. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  27. 2024RT-H: Action Hierarchies Using Languagecited by 204
  28. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 203
  29. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 172
  30. 2023MARS: An Instance-aware, Modular and Realistic Simulator for Autonomous Drivingcited by 170
  31. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 165
  32. 2023A Language Agent for Autonomous Drivingcited by 161
  33. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  34. 2023Dolphins: Multimodal Language Model for Drivingcited by 124
  35. 2023Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Drivingcited by 122
  36. 2025AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoningcited by 108
  37. 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 102
  38. 2023WoVoGen: World Volume-aware Diffusion for Controllable Multi-camera Driving Scene Generationcited by 93

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 11.

11 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Driving
  2. 2026Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Driving
  3. 2026CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Driving
  4. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future
  5. 2026WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving
  6. 2026WorldVLM: Combining World Model Forecasting and Vision-Language Reasoning
  7. 2026HyWorldVLA: A Vision-Language-Action Model with Hybrid World Modeling for Autonomous Driving
  8. 2026RAF-VLA: Representation Alignment with the Future for End-to-End Autonomous Driving