ReasonDrive: Efficient Visual Question Answering for Autonomous Vehicles with Reasoning-Enhanced Small Vision-Language Models
On the map: LLM reasoning → Chain-of-thought reasoning for driving
also in 2 more
Abstract (arXiv)
Vision-language models (VLMs) show promise for autonomous driving but often lack transparent reasoning capabilities that are critical for safety. We investigate whether explicitly modeling reasoning during fine-tuning enhances VLM performance on driving decision tasks. Using GPT-4o, we generate structured reasoning chains for driving scenarios from the DriveLM benchmark with category-specific prompting strategies. We compare reasoning-based fine-tuning, answer-only fine-tuning, and baseline instruction-tuned models across multiple small VLM families (Llama 3.2, Llava 1.5, and Qwen 2.5VL). Our results demonstrate that reasoning-based fine-tuning consistently outperforms alternatives, with Llama3.2-11B-reason achieving the highest performance. Models fine-tuned with reasoning show substantial improvements in accuracy and text generation quality, suggesting explicit reasoning enhances internal representations for driving decisions. These findings highlight the importance of transparent decision processes in safety-critical domains and offer a promising direction for developing more interpretable autonomous driving systems.
Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning. Builds on 27 of its 29 references that are in the map. First posted on arXiv 14 Apr 2025.
Builds on
27 of 29 references.
References · 27
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023Visual Instruction Tuningcited by 7,869
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2024GPT-4o System Cardcited by 4,588
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
- 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 290
- 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 262
- 2023LingoQA: Visual Question Answering for Autonomous Drivingcited by 145
- 2023Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Drivingcited by 89
15 more
- 2025HERMES: A Unified Self-Driving World Model for Simultaneous 3D Scene Understanding and Generationcited by 56
- 2024Multi-Frame, Lightweight & Efficient Vision-Language Models for Question Answering in Autonomous Drivingcited by 52
- 2024VLM-Auto: VLM-based Autonomous Driving Assistant with Human-like Behavior and Understanding for Complex Road Scenescited by 23
- 2023Explaining Autonomous Driving Actions with Visual Question Answeringcited by 16
- 2024SimpleLLM4AD: An End-to-End Vision-Language Model with Graph Visual Question Answering for Autonomous Drivingcited by 16
- 2024MiniDrive: More Efficient Vision-Language Models with Multi-Level 2D Features as Text Tokens for Autonomous Drivingcited by 13
- 2025CurricuVLM: Towards Safe Autonomous Driving via Personalized Safety-Critical Curriculum Learning with Vision-Language Modelscited by 12
- 2024Video Token Sparsification for Efficient Multimodal LLMs in Autonomous Drivingcited by 11
- 2025VLM-Assisted Continual learning for Visual Question Answering in Self-Drivingcited by 6
- 2024Query3D: LLM-Powered Open-Vocabulary Scene Segmentation with Language Embedded 3D Gaussiancited by 5
- 2024Edge-Cloud Collaborative Motion Planning for Autonomous Driving with Large Language Modelscited by 5
- 2024XLM for Autonomous Driving Systems: A Comprehensive Reviewcited by 5
- 2024Optimizing Visual Question Answering Models for Driving: Bridging the Gap Between Human and Machine Attention Patternscited by 4
- 2023Towards a performance analysis on pre-trained Visual Question Answering models for autonomous drivingcited by 3
- 2024Integrating Object Detection Modality into Visual Language Model for Enhanced Autonomous Driving Agentcited by 3
Built on by
Citing papers per year: 2025: 1; 2026: 7.
8 papers.
- 2025MMDrive: Interactive Scene Understanding Beyond Vision with Multi-representational Fusioncited by 3
- 2026How Well Do Vision-Language Models Understand Sequential Driving Scenes? A Sensitivity Studycited by 2
- 2026What's Hidden Matters: Identifying Planning-Critical Occluded Agents using Vision-Language Modelscited by 1
- 2026ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awarenesscited by 1
- 2026Drive-KD: Multi-Teacher Distillation for VLMs in Autonomous Drivingcited by 0
- 2026Learning from Human Driving: A Human-in-the-Loop Online Behavior Cloning Framework for Autonomous Drivingcited by 0
- 2026Vision-Language Assistant for Emotional Reactions to Risky Drivingcited by 0
- 2026DriveHierarchy: A Benchmark for Diagnosing VLM Driving Capabilities from Open-Loop Understanding to Closed-Loop Executioncited by 0
- 2026DriveHierarchy: A Benchmark for Diagnosing VLM Driving Capabilities from Open-Loop Understanding to Closed-Loop Execution25 Sep 2026
- 2026ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awareness26 Jul 2026
- 2026Vision-Language Assistant for Emotional Reactions to Risky Driving17 Jul 2026
- 2026What's Hidden Matters: Identifying Planning-Critical Occluded Agents using Vision-Language Models1 Jul 2026
- 2026Learning from Human Driving: A Human-in-the-Loop Online Behavior Cloning Framework for Autonomous Driving6 Jun 2026
- 2026How Well Do Vision-Language Models Understand Sequential Driving Scenes? A Sensitivity Study8 Apr 2026
- 2026Drive-KD: Multi-Teacher Distillation for VLMs in Autonomous Driving29 Jan 2026
- 2025MMDrive: Interactive Scene Understanding Beyond Vision with Multi-representational Fusion15 Dec 2025
Similar, not cited
Close in wording, no citation either way
- 2023Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Driving
- 2025Fine-Grained Evaluation of Large Vision-Language Models in Autonomous Driving
- 2025Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives
- 2025VLMs Guided Interpretable Decision Making for Autonomous Driving
- 2025DriveRX: A Vision-Language Reasoning Model for Cross-Task Autonomous Driving
- 2025DriveLMM-o1: A Step-by-Step Reasoning Dataset and Large Multimodal Model for Driving Scenario Understanding
- 2025Reasoning-VLA: A Fast and General Vision-Language-Action Reasoning Model for Autonomous Driving
- 2026Outcome-Guided Distillation: A Teacher-Student Framework to Advance VLM Reasoning in Autonomous Driving