# ReasonDrive: Efficient Visual Question Answering for Autonomous Vehicles with Reasoning-Enhanced Small Vision-Language Models

arXiv: https://arxiv.org/abs/2504.10757 · HTML: https://mlsanity.com/paper/2504.10757/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning.
- Builds on 27 of its 29 references that are in the map.
- First posted on arXiv 14 Apr 2025.

## Placed in

- LLM reasoning / [Chain-of-thought reasoning for driving](https://mlsanity.com/lr/chain-of-thought-reasoning-for-driving/)
- Multimodal LLMs / vision-language models / [Driving VLMs and open-vocabulary detection](https://mlsanity.com/mllm/driving-vlms-and-open-vocabulary-detection/)
- End-to-end autonomous driving / [VLMs for autonomous driving](https://mlsanity.com/driving/vlms-for-autonomous-driving/)

## Builds on (most cited first, 27 of 27)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [DriveLM: Driving with Graph Visual Question Answering](https://mlsanity.com/paper/2312.14150/) (2023)
- [A Survey on Multimodal Large Language Models for Autonomous Driving](https://mlsanity.com/paper/2311.12320/) (2023)
- [NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenario](https://mlsanity.com/paper/2305.14836/) (2023)
- [LingoQA: Visual Question Answering for Autonomous Driving](https://mlsanity.com/paper/2312.14115/) (2023)
- [Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Driving](https://mlsanity.com/paper/2310.02251/) (2023)
- [HERMES: A Unified Self-Driving World Model for Simultaneous 3D Scene Understanding and Generation](https://mlsanity.com/paper/2501.14729/) (2025)
- [Multi-Frame, Lightweight & Efficient Vision-Language Models for Question Answering in Autonomous Driving](https://mlsanity.com/paper/2403.19838/) (2024)
- [VLM-Auto: VLM-based Autonomous Driving Assistant with Human-like Behavior and Understanding for Complex Road Scenes](https://mlsanity.com/paper/2405.05885/) (2024)
- [Explaining Autonomous Driving Actions with Visual Question Answering](https://mlsanity.com/paper/2307.10408/) (2023)
- [SimpleLLM4AD: An End-to-End Vision-Language Model with Graph Visual Question Answering for Autonomous Driving](https://mlsanity.com/paper/2407.21293/) (2024)
- [MiniDrive: More Efficient Vision-Language Models with Multi-Level 2D Features as Text Tokens for Autonomous Driving](https://mlsanity.com/paper/2409.07267/) (2024)
- [CurricuVLM: Towards Safe Autonomous Driving via Personalized Safety-Critical Curriculum Learning with Vision-Language Models](https://mlsanity.com/paper/2502.15119/) (2025)
- [Video Token Sparsification for Efficient Multimodal LLMs in Autonomous Driving](https://mlsanity.com/paper/2409.11182/) (2024)
- [VLM-Assisted Continual learning for Visual Question Answering in Self-Driving](https://mlsanity.com/paper/2502.00843/) (2025)
- [Query3D: LLM-Powered Open-Vocabulary Scene Segmentation with Language Embedded 3D Gaussian](https://mlsanity.com/paper/2408.03516/) (2024)
- [Edge-Cloud Collaborative Motion Planning for Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2408.09972/) (2024)
- [XLM for Autonomous Driving Systems: A Comprehensive Review](https://mlsanity.com/paper/2409.10484/) (2024)
- [Optimizing Visual Question Answering Models for Driving: Bridging the Gap Between Human and Machine Attention Patterns](https://mlsanity.com/paper/2406.09203/) (2024)
- [Towards a performance analysis on pre-trained Visual Question Answering models for autonomous driving](https://mlsanity.com/paper/2307.09329/) (2023)
- [Integrating Object Detection Modality into Visual Language Model for Enhanced Autonomous Driving Agent](https://mlsanity.com/paper/2411.05898/) (2024)

## Built on by (most cited first, 8 of 8)

- [MMDrive: Interactive Scene Understanding Beyond Vision with Multi-representational Fusion](https://mlsanity.com/paper/2512.13177/) (2025)
- [How Well Do Vision-Language Models Understand Sequential Driving Scenes? A Sensitivity Study](https://mlsanity.com/paper/2604.06750/) (2026)
- [What's Hidden Matters: Identifying Planning-Critical Occluded Agents using Vision-Language Models](https://mlsanity.com/paper/2607.00283/) (2026)
- [ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awareness](https://mlsanity.com/paper/2607.23537/) (2026)
- [Drive-KD: Multi-Teacher Distillation for VLMs in Autonomous Driving](https://mlsanity.com/paper/2601.21288/) (2026)
- [Learning from Human Driving: A Human-in-the-Loop Online Behavior Cloning Framework for Autonomous Driving](https://mlsanity.com/paper/2606.08170/) (2026)
- [Vision-Language Assistant for Emotional Reactions to Risky Driving](https://mlsanity.com/paper/2607.16181/) (2026)
- [DriveHierarchy: A Benchmark for Diagnosing VLM Driving Capabilities from Open-Loop Understanding to Closed-Loop Execution](https://mlsanity.com/paper/2609.31814/) (2026)

## Abstract (arXiv)

Vision-language models (VLMs) show promise for autonomous driving but often lack transparent reasoning capabilities that are critical for safety. We investigate whether explicitly modeling reasoning during fine-tuning enhances VLM performance on driving decision tasks. Using GPT-4o, we generate structured reasoning chains for driving scenarios from the DriveLM benchmark with category-specific prompting strategies. We compare reasoning-based fine-tuning, answer-only fine-tuning, and baseline instruction-tuned models across multiple small VLM families (Llama 3.2, Llava 1.5, and Qwen 2.5VL). Our results demonstrate that reasoning-based fine-tuning consistently outperforms alternatives, with Llama3.2-11B-reason achieving the highest performance. Models fine-tuned with reasoning show substantial improvements in accuracy and text generation quality, suggesting explicit reasoning enhances internal representations for driving decisions. These findings highlight the importance of transparent decision processes in safety-critical domains and offer a promising direction for developing more interpretable autonomous driving systems.