Learning Vision-Language-Action World Models for Autonomous Driving
On the map: World models → World models for autonomous driving
also in 2 more
Abstract (arXiv)
Vision-Language-Action (VLA) models have recently achieved notable progress in end-to-end autonomous driving by integrating perception, reasoning, and control within a unified multimodal framework. However, they often lack explicit modeling of temporal dynamics and global world consistency, which limits their foresight and safety. In contrast, world models can simulate plausible future scenes but generally struggle to reason about or evaluate the imagined future they generate. In this work, we present VLA-World, a simple yet effective VLA world model that unifies predictive imagination with reflective reasoning to improve driving foresight. VLA-World first uses an action-derived feasible trajectory to guide the generation of the next-frame image, capturing rich spatial and temporal cues that describe how the surrounding environment evolves. The model then reasons over this self-generated future imagined frame to refine the predicted trajectory, achieving higher performance and better interpretability. To support this pipeline, we curate nuScenes-GR-20K, a generative reasoning dataset derived from nuScenes, and employ a three-stage training strategy that includes pretraining, supervised fine-tuning, and reinforcement learning. Extensive experiments demonstrate that VLA-World consistently surpasses state-of-the-art VLA and world-model baselines on both planning and future-generation benchmarks. Project page: https://vlaworld.github.io
Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 82 of its 87 references that are in the map. First posted on arXiv 10 Apr 2026.
Builds on
82 of 87 references.
Closest ancestors · 16
Same direction, newest first
- 2025OmniNWM: Omniscient Driving Navigation World Models21 Oct
- 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model1 Jul
- 2025ReSim: Reliable World Simulation for Autonomous Driving11 Jun
- 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning6 Apr
- 2025Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving11 Feb
- 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT27 Dec
- 2024Doe-1: Closed-Loop Autonomous Driving with Large World Model12 Dec
- 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving26 Aug
- 2024DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Driving7 May
- 2024GenAD: Generalized Predictive Model for Autonomous Driving14 Mar
- 2024GenAD: Generative End-to-End Autonomous Driving18 Feb
- 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
4 more
Other references · 66
- 2023GPT-4 Technical Reportcited by 18,132
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Visual Instruction Tuningcited by 7,792
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2017Neural Discrete Representation Learningcited by 4,650
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
38 more
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2024Qwen2 Technical Reportcited by 2,076
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
- 2022Planning-oriented Autonomous Drivingcited by 1,068
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
- 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
- 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
- 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
- 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
- 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
- 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
- 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 287
- 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
- 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
- 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
- 2024RT-H: Action Hierarchies Using Languagecited by 204
- 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 203
- 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 172
- 2023MARS: An Instance-aware, Modular and Realistic Simulator for Autonomous Drivingcited by 170
- 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 165
- 2023A Language Agent for Autonomous Drivingcited by 161
- 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
- 2023Dolphins: Multimodal Language Model for Drivingcited by 124
- 2023Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Drivingcited by 122
- 2025AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoningcited by 108
- 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 102
- 2023WoVoGen: World Volume-aware Diffusion for Controllable Multi-camera Driving Scene Generationcited by 93
50 most cited of 66 All 66 (JSON)
Built on by
Citing papers per year: 2026: 11.
11 papers.
- 2026Driving Intents Amplify Planning-Oriented Reinforcement Learningcited by 1
- 20264D-WAM: 4D Consistent World Modeling for Autonomous Drivingcited by 1
- 2026World-Action Models for Robot Learning and Control: A Surveycited by 1
- 2026Multi-Agent Embodied Autonomous Driving (MAEAD): From V2X Information Exchange to Shared World Modelscited by 0
- 2026SPVC: Structured and Panoptic Video Fixing for Cross-Dataset Driving Scene Renderingcited by 0
- 2026A Collaborative Multi-Modality Interaction for VLA-based End-to-End Autonomous Drivingcited by 0
- 2026Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Drivingcited by 0
- 2026CoVLM-Bench: A Real-World Benchmark for Cooperative Driving Question Answering and Planningcited by 0
- 2026V2X-WAM: A Cooperative World Action Model for End-to-End Autonomous Drivingcited by 0
- 2026ReWAM: Reciprocal World Action Models for Interactive Autonomous Drivingcited by 0
- 2026AffordDrive3D: Affordance-Aware World-Action Modeling with Spatial Understandingcited by 0
- 2026AffordDrive3D: Affordance-Aware World-Action Modeling with Spatial Understanding8 Oct 2026
- 2026ReWAM: Reciprocal World Action Models for Interactive Autonomous Driving30 Sep 2026
- 2026V2X-WAM: A Cooperative World Action Model for End-to-End Autonomous Driving29 Sep 2026
- 2026CoVLM-Bench: A Real-World Benchmark for Cooperative Driving Question Answering and Planning22 Sep 2026
- 2026World-Action Models for Robot Learning and Control: A Survey13 Sep 2026
- 2026Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Driving31 Aug 2026
- 2026A Collaborative Multi-Modality Interaction for VLA-based End-to-End Autonomous Driving21 Aug 2026
- 2026SPVC: Structured and Panoptic Video Fixing for Cross-Dataset Driving Scene Rendering18 Aug 2026
- 20264D-WAM: 4D Consistent World Modeling for Autonomous Driving10 Aug 2026
- 2026Multi-Agent Embodied Autonomous Driving (MAEAD): From V2X Information Exchange to Shared World Models11 Jun 2026
- 2026Driving Intents Amplify Planning-Oriented Reinforcement Learning12 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Driving
- 2026Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Driving
- 2026CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Driving
- 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future
- 2026WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving
- 2026WorldVLM: Combining World Model Forecasting and Vision-Language Reasoning
- 2026HyWorldVLA: A Vision-Language-Action Model with Hybrid World Modeling for Autonomous Driving
- 2026RAF-VLA: Representation Alignment with the Future for End-to-End Autonomous Driving