mlsanityData to 7 Oct 2026

VLMs for autonomous driving

270 papers1 this weekHigh confidence in this grouping: the papers form one clear topicHigh confidence in this grouping: the papers form one clear topic · About

Keywords: driving, autonomous, autonomous driving, vlms, reasoning

Key facts. VLMs for autonomous driving: 270 papers on arXiv since 2021 (data through 7 Oct 2026). 104 of its 270 mapped papers (39%) cite “DriveLM: Driving with Graph Visual Question Answering” (2023, arXiv 2312.14150). 82 of its 270 mapped papers (30%) cite “DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models” (2024, arXiv 2402.12289). 8 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

8 new papers, 17 Sep – 7 Oct; 1 this week.

  1. This week · 1
  2. Transferable Spatial Temporal Coherence Adversarial Attack on Black-Box Vision Language Models for Autonomous Driving also fits: Adversarial scenarios and safety testing
  3. Earlier · 7
  4. Localized Conformal Safety Monitoring with Vision-Language Models for Autonomous Driving also fits: End-to-end system architecture
  5. Towards Reliable Vision-Language Models for Autonomous Driving
  6. Beyond Retrieval Relevance: Scene-Grounded Risk Entailment for Vision-Language Driving also fits: Reasoning VLA and VLM driving policies
  7. FocusDrive: Reasoning with Visual Focus for Autonomous Driving also fits: Reasoning VLA and VLM driving policies
  8. RCVLA: 4D Radar-Grounded Semantic Reasoning and Trajectory Arbitration for Autonomous Driving also fits: Reasoning VLA and VLM driving policies
  9. From Scene Graphs to Answers: Selective Neuro-Symbolic Reasoning for Autonomous Driving also fits: Reasoning VLA and VLM driving policies
  10. CoVLM-Bench: A Real-World Benchmark for Cooperative Driving Question Answering and Planning

Foundations

Most cited by papers here

  1. DriveLM: Driving with Graph Visual Question Answering2023cited by 104 here
  2. DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models2024cited by 82 here
  3. VAD: Vectorized Scene Representation for Efficient Autonomous Driving2023cited by 41 here
  4. LingoQA: Visual Question Answering for Autonomous Driving2023cited by 39 here
  5. Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives2025cited by 37 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 31 here
  2. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 41 here
  3. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 39 here
  4. 2023Language Prompt for Autonomous Drivingcited by 26 here
  5. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 83 here
  6. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 53 here
  7. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 38 here
  8. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 104 here
  9. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 82 here
  10. 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Drivingcited by 34 here
  11. 2025Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectivescited by 37 here
  12. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 23 here
12 more it builds on, not in this chain
  1. 2023DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Modelscited by 26 here
  2. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Drivingcited by 45 here
  3. 2023Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Drivingcited by 23 here
  4. 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 28 here
  5. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 25 here
  6. 2023Dolphins: Multimodal Language Model for Drivingcited by 26 here
  7. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 23 here
  8. 2023Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Drivingcited by 31 here
  9. 2023LingoQA: Visual Question Answering for Autonomous Drivingcited by 39 here
  10. 2024Holistic Autonomous Driving Understanding by Bird's-Eye-View Injected Multi-Modal Large Modelscited by 23 here
  11. 2024VLP: Vision Language Planning for Autonomous Drivingcited by 28 here
  12. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 30 here

The 24 most cited of 432.

Browse

30 of 270, ranked 6–35

  1. 6Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Driving202331
  2. 7Vision Language Models in Autonomous Driving: A Survey and Outlook202328
  3. 8Language Prompt for Autonomous Driving202326
  4. 9Dolphins: Multimodal Language Model for Driving202326
  5. 10Holistic Autonomous Driving Understanding by Bird's-Eye-View Injected Multi-Modal Large Models202423
  6. 11Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Driving202323
  7. 12Multi-Frame, Lightweight & Efficient Vision-Language Models for Question Answering in Autonomous Driving202422
  8. 13Automated Evaluation of Large Vision-Language Models on Self-driving Corner Cases202420
  9. 14VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervision202417
  10. 15Embodied Understanding of Driving Scenarios202413
All 30
  1. 16HiLM-D: Enhancing MLLMs with Multi-Scale High-Resolution Details for Autonomous Driving202313
  2. 17CarLLaVA: Vision language models for camera-only closed-loop driving202412
  3. 18NuPlanQA: A Large-Scale Dataset and Benchmark for Multi-View Driving Scene Understanding in Multi-Modal Large Language Models202510
  4. 19NuScenes-MQA: Integrated Evaluation of Captions and QA for Autonomous Driving Datasets using Markup Annotations202310
  5. 20Fine-Grained Evaluation of Large Vision-Language Models in Autonomous Driving20259
  6. 21Can LVLMs Obtain a Driver's License? A Benchmark Towards Reliable AGI for Autonomous Driving20249
  7. 22ReasonNet: End-to-End Driving with Temporal and Global Reasoning20239
  8. 23SimpleLLM4AD: An End-to-End Vision-Language Model with Graph Visual Question Answering for Autonomous Driving20248
  9. 24MiniDrive: More Efficient Vision-Language Models with Multi-Level 2D Features as Text Tokens for Autonomous Driving20248
  10. 25On the Road with GPT-4V(ision): Early Explorations of Visual-Language Model on Autonomous Driving20238
  11. 26NEAT: Neural Attention Fields for End-to-End Autonomous Driving20218
  12. 27NuScenes-SpatialQA: A Spatial Understanding and Reasoning Benchmark for Vision-Language Models in Autonomous Driving20257
  13. 28Are Vision LLMs Road-Ready? A Comprehensive Benchmark for Safety-Critical Driving Video Understanding20257
  14. 29STSBench: A Spatio-temporal Scenario Benchmark for Multi-modal Large Language Models in Autonomous Driving20257
  15. 30DME-Driver: Integrating Human Decision Logic and 3D Scene Perception in Autonomous Driving20246
  16. 31VLM-Auto: VLM-based Autonomous Driving Assistant with Human-like Behavior and Understanding for Complex Road Scenes20246
  17. 32AutoTrust: Benchmarking Trustworthiness in Large Vision Language Models for Autonomous Driving20246
  18. 33STRIDE-QA: Visual Question Answering Dataset for Spatiotemporal Reasoning in Urban Driving Scenes20255
  19. 34SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving20255
  20. 35SURDS: Benchmarking Spatial Understanding and Reasoning in Driving Scenarios with Vision Language Models20245