mlsanityData to 14 Oct 2026

A Survey on Vision-Language-Action Models for Autonomous Driving

arXiv 2506.24044 ↗First posted 30 Jun 2025cs.CV cs.AI cs.ROcited by 72

On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection

also in 2 more
Abstract (arXiv)

The rapid progress of multimodal large language models (MLLM) has paved the way for Vision-Language-Action (VLA) paradigms, which integrate visual perception, natural language understanding, and control within a single policy. Researchers in autonomous driving are actively adapting these methods to the vehicle domain. Such models promise autonomous vehicles that can interpret high-level instructions, reason about complex traffic scenes, and make their own decisions. However, the literature remains fragmented and is rapidly expanding. This survey offers the first comprehensive overview of VLA for Autonomous Driving (VLA4AD). We (i) formalize the architectural building blocks shared across recent work, (ii) trace the evolution from early explainer to reasoning-centric VLA models, and (iii) compare over 20 representative models according to VLA's progress in the autonomous driving domain. We also consolidate existing datasets and benchmarks, highlighting protocols that jointly measure driving safety, accuracy, and explanation quality. Finally, we detail open challenges - robustness, real-time efficiency, and formal verification - and outline future directions of VLA4AD. This survey provides a concise yet complete reference for advancing interpretable socially aligned autonomous vehicles. Github repo is available at hrefhttps://github.com/JohnsonJiang1996/Awesome-VLA4ADSicongJiang/Awesome-VLA4AD.

Key facts. Cited by 72 papers in the map (data through 14 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 156 of its 183 references that are in the map. First posted on arXiv 30 Jun 2025.

Builds on

156 of 183 references.

Closest ancestors · 28

Same direction, newest first

  1. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving9 Jun
  2. 2025S4-Driver: Scalable Self-Supervised Driving Multimodal Large Language Modelwith Spatio-Temporal Visual Representation30 May
  3. 2025ReasonPlan: Unified Scene Prediction and Decision Reasoning for Closed-loop Autonomous Driving26 May
  4. 2025AgentThink: A Unified Framework for Tool-Augmented Chain-of-Thought Reasoning in Vision-Language Models for Autonomous Driving21 May
  5. 2025TS-VLM: Text-Guided SoftSort Pooling for Vision-Language Models in Multi-View Driving Reasoning19 May
  6. 2025Extending Large Vision-Language Model for Diverse Interactive Tasks in Autonomous Driving13 May
  7. 2025ChatBEV: Empowering Traffic Scene Understanding and Simulation via Vision-Language Model18 Mar
  8. 2025DynRsl-VLM: Enhancing Autonomous Driving Perception with Dynamic Resolution Vision-Language Models14 Mar
  9. 2025DriveLMM-o1: A Step-by-Step Reasoning Dataset and Large Multimodal Model for Driving Scenario Understanding13 Mar
  10. 2025INSIGHT: Enhancing Autonomous Driving Safety through Vision-Language Models on Context-Aware Hazard Detection and Edge Case Evaluation1 Feb
  11. 2025Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives7 Jan
  12. 2024VLM-RL: A Unified Vision Language Models and Reinforcement Learning Framework for Safe Autonomous Driving20 Dec
16 more
  1. 2024VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervision19 Dec
  2. 2024World knowledge-enhanced Reasoning Using Instruction-guided Interactor in Autonomous Driving9 Dec
  3. 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving29 Oct
  4. 2024MiniDrive: More Efficient Vision-Language Models with Multi-Level 2D Features as Text Tokens for Autonomous Driving11 Sep
  5. 2024OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Driving5 Sep
  6. 2024CarLLaVA: Vision language models for camera-only closed-loop driving14 Jun
  7. 2024LeGo-Drive: Language-enhanced Goal-oriented Closed-Loop End-to-End Autonomous Driving29 Mar
  8. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models19 Feb
  9. 2024RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model16 Feb
  10. 2023LingoQA: Visual Question Answering for Autonomous Driving21 Dec
  11. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving14 Dec
  12. 2023Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Driving6 Dec
  13. 2023Dolphins: Multimodal Language Model for Driving1 Dec
  14. 2023A Survey on Multimodal Large Language Models for Autonomous Driving21 Nov
  15. 2023Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Driving3 Oct
  16. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model2 Oct

Other references · 128

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  8. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  9. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  10. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  11. 2023Visual Instruction Tuningcited by 7,869
  12. 2025Qwen2.5-VL Technical Reportcited by 5,587
38 more
  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  2. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  3. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  5. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  6. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  7. 2022Planning-oriented Autonomous Drivingcited by 1,087
  8. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 883
  9. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
  10. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 712
  11. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
  12. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 665
  13. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  14. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 519
  15. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 490
  16. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
  17. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 375
  18. 2023GPT-Driver: Learning to Drive with GPTcited by 347
  19. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 334
  20. 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 328
  21. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 305
  22. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 289
  23. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  24. 2018Learning to Drive in a Daycited by 273
  25. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 260
  26. 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 254
  27. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 252
  28. 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 249
  29. 2021End-to-end Interpretable Neural Motion Plannercited by 242
  30. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 241
  31. 2020Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Netcited by 228
  32. 2018Textual Explanations for Self-Driving Vehiclescited by 226
  33. 2024GenAD: Generative End-to-End Autonomous Drivingcited by 225
  34. 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 212
  35. 2023DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Modelscited by 207
  36. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  37. 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 178
  38. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 176

50 most cited of 128 All 128 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 20; 2026: 52.

72 papers.

Sort:

By area:

  1. 2025AdaThinkDrive: Adaptive Thinking via Reinforcement Learning for Autonomous Drivingcited by 58
  2. 2025VLSA: Vision-Language-Action Models with Plug-and-Play Safety Constraint Layercited by 34
  3. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 33
  4. 2025Efficient Vision-Language-Action Models for Embodied Manipulation: A Systematic Surveycited by 26
  5. 2025Mixture of Horizons in Action Chunkingcited by 21
  6. 2026Unleashing VLA Potentials in Autonomous Driving via Explicit Learning from Failurescited by 19
  7. 2026VideoSeek: Long-Horizon Video Agent with Tool-Guided Seekingcited by 17
  8. 2025ResAD: Normalized Residual Trajectory Modeling for End-to-End Autonomous Drivingcited by 14
  9. 2026Vision-Language-Action Safety: Threats, Challenges, Evaluations, and Mechanismscited by 13
  10. 2025EvaDrive: Evolutionary Adversarial Policy Optimization for End-to-End Autonomous Drivingcited by 12
  11. 2025AirSim360: A Panoramic Simulation Platform within Drone Viewcited by 12
  12. 2026ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Drivingcited by 12
38 more
  1. 2026Goal2Skill: Long-Horizon Manipulation with Adaptive Planning and Reflectioncited by 11
  2. 2025Research Challenges and Progress in the End-to-End V2X Cooperative Autonomous Driving Competitioncited by 9
  3. 2025MTRDrive: Memory-Tool Synergistic Reasoning for Robust Autonomous Driving in Corner Casescited by 9
  4. 2026On the Generalization Capacities of MLLMs for Spatial Intelligencecited by 7
  5. 2026CausalVAD: De-confounding End-to-End Autonomous Driving via Causal Interventioncited by 7
  6. 2025SafePLUG: Empowering Multimodal LLMs with Pixel-Level Insight and Temporal Grounding for Traffic Accident Understandingcited by 6
  7. 2025RoaD: Rollouts as Demonstrations for Closed-Loop Supervised Fine-Tuning of Autonomous Driving Policiescited by 6
  8. 2025VDRive: Leveraging Reinforced VLA and Diffusion Policy for End-to-end Autonomous Drivingcited by 5
  9. 2026Drive My Way: Preference Alignment of Vision-Language-Action Model for Personalized Drivingcited by 5
  10. 2026Open-Loop Planning, Closed-Loop Verification: Speculative Verification for VLAcited by 5
  11. 2025Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systemscited by 4
  12. 2026UAV-Track VLA: Embodied Aerial Tracking via Vision-Language-Action Modelscited by 4
  13. 2025Multi-Context Fusion Transformer for Pedestrian Crossing Intention Prediction in Urban Environmentscited by 3
  14. 2025Foundation Models for Trajectory Planning in Autonomous Driving: A Review of Progress and Open Challengescited by 3
  15. 2026RAD-LAD: Rule and Language Grounded Autonomous Driving in Real-Timecited by 3
  16. 2026Judge, Then Drive: A Critic-Centric Vision Language Action Framework for Autonomous Drivingcited by 3
  17. 2026Bench2Drive-Robust: Benchmarking Closed-Loop Autonomous Driving under Deployment Perturbationscited by 3
  18. 2025SafeCoop: Unravelling Full Stack Safety in Agentic Collaborative Drivingcited by 2
  19. 2025E3AD: An Emotion-Aware Vision-Language-Action Model for Human-Centric End-to-End Autonomous Drivingcited by 2
  20. 2026Listen, Look, Drive: Coupling Audio Instructions for User-aware VLA-based Autonomous Drivingcited by 2
  21. 2026AppleVLM: End-to-end Autonomous Driving with Advanced Perception and Planning-Enhanced Vision-Language Modelscited by 2
  22. 2026WalkGPT: Grounded Vision-Language Conversation with Depth-Aware Segmentation for Pedestrian Navigationcited by 2
  23. 2026Annotation-Free Detection of Drivable Areas and Curbs Leveraging LiDAR Point Cloud Mapscited by 2
  24. 2026EgoDyn-Bench: Evaluating Ego-Motion Understanding in Vision-Centric Foundation Models for Autonomous Drivingcited by 2
  25. 2026PriorEye: Geospatial Visual Priors for End-to-End Autonomous Drivingcited by 2
  26. 2026CLEAR: Closed-Loop Reinforcement Learning at Scale for End-to-End Autonomous Drivingcited by 2
  27. 2026Post-Training in End-to-End Autonomous Drivingcited by 2
  28. 2026From Representational Complementarity to Dual Systems: Synergizing VLM and Vision-Only Backbones for End-to-End Drivingcited by 1
  29. 2026Sim2Real-AD: A Modular Sim-to-Real Framework for Deploying VLM-Guided Reinforcement Learning in Real-World Autonomous Drivingcited by 1
  30. 2026DriveVer: Lightweight Trajectory Evaluator as Test-Time Verifier for Autonomous Drivingcited by 1
  31. 2026WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Drivingcited by 1
  32. 2026Lights, Camera, Malfunction: When Illumination Robustness Leaves VLA Models Blind to Colorcited by 1
  33. 2026CosFly-VLA: A Spatially Aware Vision-Language-Action Model for UAV Trackingcited by 1
  34. 2026Uncovering and Mitigating Positional Blind Spots in Vision-Language-Action Modelscited by 1
  35. 2026Continuous Actions from Discrete Minds: Latent-Aligned Planning for End-to-End Autonomous Drivingcited by 1
  36. 2025MLPerf Automotivecited by 0
  37. 2025SPOT!: Map-Guided LLM Agent for Unsupervised Multi-CCTV Dynamic Object Trackingcited by 0
  38. 2026LinMU: Multimodal Understanding Made Linearcited by 0

50 most cited of 72 All 72 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Rethinking Language's Role in Efficient VLA for Autonomous Vehicles: Toward Smarter, Trustworthy Driving
  2. 2026RT-VLA: Real-Time Vision-Language-Action Models via Knowledge Distillation
  3. 2025Reasoning-VLA: A Fast and General Vision-Language-Action Reasoning Model for Autonomous Driving
  4. 2023Vision Language Models in Autonomous Driving: A Survey and Outlook
  5. 2025LightEMMA: A Longitudinal Evaluation of Vision-Language Models for Autonomous Driving
  6. 2026Unifying Language-Action Understanding and Generation for Autonomous Driving
  7. 2026SteerVLA: Steering Vision-Language-Action Models in Long-Tail Driving Scenarios
  8. 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving