mlsanityData to 14 Oct 2026

Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future

arXiv 2512.16760 ↗First posted 18 Dec 2025cs.ROcited by 33

On the map: Vision-language-action models → VLA for autonomous driving

also in 1 more
Abstract (arXiv)

Autonomous driving has long relied on modular "Perception-Decision-Action" pipelines, where hand-crafted interfaces and rule-based components often break down in complex or long-tailed scenarios. Their cascaded design further propagates perception errors, degrading downstream planning and control. Vision-Action (VA) models address some limitations by learning direct mappings from visual inputs to actions, but they remain opaque, sensitive to distribution shifts, and lack structured reasoning or instruction-following capabilities. Recent progress in Large Language Models (LLMs) and multimodal learning has motivated the emergence of Vision-Language-Action (VLA) frameworks, which integrate perception with language-grounded decision making. By unifying visual understanding, linguistic reasoning, and actionable outputs, VLAs offer a pathway toward more interpretable, generalizable, and human-aligned driving policies. This work provides a structured characterization of the emerging VLA landscape for autonomous driving. We trace the evolution from early VA approaches to modern VLA frameworks and organize existing methods into two principal paradigms: End-to-End VLA, which integrates perception, reasoning, and planning within a single model, and Dual-System VLA, which separates slow deliberation (via VLMs) from fast, safety-critical execution (via planners). Within these paradigms, we further distinguish subclasses such as textual vs. numerical action generators and explicit vs. implicit guidance mechanisms. We also summarize representative datasets and benchmarks for evaluating VLA-based driving systems and highlight key challenges and open directions, including robustness, interpretability, and instruction fidelity. Overall, this work aims to establish a coherent foundation for advancing human-compatible autonomous driving systems.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Placed in VLA for autonomous driving, one of 13 directions in Vision-language-action models. Builds on 301 of its 337 references that are in the map. First posted on arXiv 18 Dec 2025.

Builds on

301 of 337 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025Latent Chain-of-Thought World Modeling for End-to-End Driving11 Dec
  2. 2025Reasoning-VLA: A Fast and General Vision-Language-Action Reasoning Model for Autonomous Driving25 Nov
  3. 2025Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail30 Oct
  4. 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving14 Oct
  5. 2025Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Driving24 Sep
  6. 2025OccVLA: Vision-Language-Action Model with Implicit 3D Occupancy Supervision6 Sep
  7. 2025FastDriveVLA: Efficient End-to-End Driving via Plug-and-Play Reconstruction-based Token Pruning31 Jul
  8. 2025A Survey on Vision-Language-Action Models for Autonomous Driving30 Jun
  9. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning16 Jun
  10. 2025DriveAction: A Benchmark for Exploring Human-like Driving Decisions in VLA Models6 Jun
  11. 2025Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Models29 May
  12. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving23 May
2 more
  1. 2025DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Driving22 May
  2. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Model30 Mar

Other references · 287

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2024The Llama 3 Herd of Modelscited by 14,715
  7. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  9. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  10. 2025Qwen3 Technical Reportcited by 8,820
  11. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  12. 2023Visual Instruction Tuningcited by 7,869
38 more
  1. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2025Qwen2.5-VL Technical Reportcited by 5,587
  5. 2024Qwen2.5 Technical Reportcited by 4,942
  6. 2017Neural Discrete Representation Learningcited by 4,712
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  8. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  10. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  11. 2022A ConvNet for the 2020scited by 3,579
  12. 2023Qwen Technical Reportcited by 3,344
  13. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  14. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  15. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  16. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  17. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  18. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  19. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  20. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  21. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  22. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  23. 2018World Modelscited by 1,513
  24. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  25. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  26. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
  27. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  28. 2022Planning-oriented Autonomous Drivingcited by 1,087
  29. 2016End to End Learning for Self-Driving Carscited by 1,009
  30. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  31. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  32. 2022Autoregressive Image Generation using Residual Quantizationcited by 722
  33. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
  34. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
  35. 2024PaliGemma: A versatile 3B VLM for transfercited by 696
  36. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 691
  37. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 640
  38. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637

50 most cited of 287 All 287 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 32.

33 papers.

Sort:

By area:

  1. 2026VGGDrive: Empowering Vision-Language Models with Cross-View Geometric Grounding for Autonomous Drivingcited by 20
  2. 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 14
  3. 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representationcited by 13
  4. 2026Vision-Language-Action Safety: Threats, Challenges, Evaluations, and Mechanismscited by 13
  5. 2026Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Drivingcited by 11
  6. 2026Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanationcited by 10
  7. 2026nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Drivingcited by 9
  8. 2026On the Adversarial Robustness of Large Vision-Language Models under Visual Token Compressioncited by 6
  9. 2026LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generationcited by 6
  10. 2025Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systemscited by 4
  11. 2026From Articulated Kinematics to Routed Visual Control for Action-Conditioned Surgical Video Generationcited by 4
  12. 2026MapTab: A Diagnostic Benchmark for Long-Horizon Multi-Criteria Multimodal Reasoning on Heterogeneous Topological Graphscited by 3
21 more
  1. 2026DynFlowDrive: Flow-Based Dynamic World Modeling for Autonomous Drivingcited by 2
  2. 2026DLWM: Dual Latent World Models enable Holistic Gaussian-centric Pre-training in Autonomous Drivingcited by 2
  3. 2026Towards Safe Mobility: A Unified Transportation Foundation Model enabled by Open-Ended Vision-Language Datasetcited by 2
  4. 2026SafeAlign-VLA: A Negative-Enhanced Safe Alignment Framework for Risk-Aware Autonomous Drivingcited by 2
  5. 2026EventDrive: Event Cameras for Vision-Language Driving Intelligencecited by 2
  6. 2026Post-Training in End-to-End Autonomous Drivingcited by 2
  7. 2026From Representational Complementarity to Dual Systems: Synergizing VLM and Vision-Only Backbones for End-to-End Drivingcited by 1
  8. 2026CLAP: Contrastive Latent-space Prompt Optimization for End-to-end Autonomous Drivingcited by 1
  9. 2026Grounding Driving VLA via Inverse Kinematicscited by 1
  10. 2026DriveVer: Lightweight Trajectory Evaluator as Test-Time Verifier for Autonomous Drivingcited by 1
  11. 2026HyWorldVLA: A Vision-Language-Action Model with Hybrid World Modeling for Autonomous Drivingcited by 1
  12. 2026CMU-Drive and V2V-VLA: Cooperative Multi-agent Unified Driving with Reasoning Benchmark and Vehicle-to-Vehicle Vision-Language-Action Modelscited by 1
  13. 2026Looking and Listening Inside and Outside: Multimodal Artificial Intelligence Systems for Driver Safety Assessment and Intelligent Vehicle Decision-Makingcited by 0
  14. 2026From Steering to Pedalling: Do Autonomous Driving VLMs Generalize to Cyclist-Assistive Spatial Perception and Planning?cited by 0
  15. 2026Safety Case Patterns for VLA-based driving systems: Insights from SimLingocited by 0
  16. 2026LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Modelcited by 0
  17. 2026Driving on Memorycited by 0
  18. 2026Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Drivingcited by 0
  19. 2026CrashDiffuser: VLM-Guided Collision Intent Reasoning for Fine-Grained Safety-Critical Traffic Scenario Generationcited by 0
  20. 2026Explicit Geometric Chain-of-Thought for Vision-Language-Action in Autonomous Drivingcited by 0
  21. 2026VITAL: From Visual Motion to Precise Actioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Judge, Then Drive: A Critic-Centric Vision Language Action Framework for Autonomous Driving
  2. 2026Learning Vision-Language-Action World Models for Autonomous Driving
  3. 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving
  4. 2026Unifying Language-Action Understanding and Generation for Autonomous Driving
  5. 2026Teaching Vision-Language-Action Models What to See and Where to Look
  6. 2026Does Visual Information Play a Decisive Role in Vision-Language-Action Model Driving Behavior?
  7. 2026RT-VLA: Real-Time Vision-Language-Action Models via Knowledge Distillation
  8. 2025Foundation Models for Trajectory Planning in Autonomous Driving: A Review of Progress and Open Challenges