From Steering to Pedalling: Do Autonomous Driving VLMs Generalize to Cyclist-Assistive Spatial Perception and Planning?
On the map: End-to-end autonomous driving → VLMs for autonomous driving
Abstract (arXiv)
Cyclists often encounter safety-critical situations in urban traffic, highlighting the need for assistive systems that support safe and informed decision-making. Recently, vision-language models (VLMs) have demonstrated strong performance on autonomous driving benchmarks, suggesting their potential for general traffic understanding and navigation-related reasoning. However, existing evaluations are predominantly vehicle-centric and fail to assess perception and reasoning from a cyclist-centric viewpoint. To address this gap, we introduce CyclingVQA, a diagnostic benchmark designed to probe perception, spatio-temporal understanding, and traffic-rule-to-lane reasoning from a cyclist's perspective. Evaluating 31+ recent VLMs spanning general-purpose, spatially enhanced, and autonomous-driving-specialized models, we find that current models demonstrate encouraging capabilities, while also revealing clear areas for improvement in cyclist-centric perception and reasoning, particularly in interpreting cyclist-specific traffic cues and associating signs with the correct navigational lanes. Notably, several driving-specialized models underperform strong generalist VLMs, indicating limited transfer from vehicle-centric training to cyclist-assistive scenarios. Finally, through systematic error analysis, we identify recurring failure modes to guide the development of more effective cyclist-assistive intelligent systems.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in VLMs for autonomous driving, one of 9 directions in End-to-end autonomous driving. Builds on 51 of its 66 references that are in the map. First posted on arXiv 11 Feb 2026.
Builds on
51 of 66 references.
Closest ancestors · 9
Same direction, newest first
- 2025DriveQA: Passing the Driving Knowledge Test29 Aug
- 2025NuScenes-SpatialQA: A Spatial Understanding and Reasoning Benchmark for Vision-Language Models in Autonomous Driving4 Apr
- 2025Fine-Grained Evaluation of Large Vision-Language Models in Autonomous Driving27 Mar
- 2025Retrieval-Based Interleaved Visual Chain-of-Thought in Real-World Driving Scenarios8 Jan
- 2025Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives7 Jan
- 2024AutoTrust: Benchmarking Trustworthiness in Large Vision Language Models for Autonomous Driving19 Dec
- 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models19 Feb
- 2023Dolphins: Multimodal Language Model for Driving1 Dec
- 2023Vision Language Models in Autonomous Driving: A Survey and Outlook22 Oct
Other references · 42
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
- 2025Gemma 3 Technical Reportcited by 1,686
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
30 more
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
- 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
- 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
- 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 297
- 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
- 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoningcited by 245
- 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
- 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 172
- 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 159
- 2023LLM4Drive: A Survey of Large Language Models for Autonomous Drivingcited by 136
- 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmarkcited by 133
- 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Groundingcited by 132
- 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Modelscited by 111
- 2025AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoningcited by 108
- 2025OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Modelscited by 105
- 2025Ovis2.5 Technical Reportcited by 103
- 2025PerceptionLM: Open-Access Data and Models for Detailed Visual Understandingcited by 88
- 2025Visual Spatial Tuningcited by 85
- 2025Scaling Spatial Intelligence with Multimodal Foundation Modelscited by 75
- 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
- 2025SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoningcited by 65
- 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 62
- 2024GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Taskscited by 50
- 2025DriveLMM-o1: A Step-by-Step Reasoning Dataset and Large Multimodal Model for Driving Scenario Understandingcited by 35
- 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
- 2024RoboTron-Drive: All-in-One Large Multimodal Model for Autonomous Drivingcited by 30
- 2025Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMscited by 30
- 2024ROADWork: A Dataset and Benchmark for Learning to Recognize, Observe, Analyze and Drive Through Work Zonescited by 20
- 2025SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Datacited by 19
- 2025FoundationMotion: Auto-Labeling and Reasoning about Spatial Movement in Videoscited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Are VLMs Ready for Lane Topology Awareness in Autonomous Driving?
- 2026GeoDrive-Bench: Benchmarking Region-Specific Multimodal Reasoning in Autonomous Driving
- 2026DriveHierarchy: A Benchmark for Diagnosing VLM Driving Capabilities from Open-Loop Understanding to Closed-Loop Execution
- 2025LightEMMA: A Longitudinal Evaluation of Vision-Language Models for Autonomous Driving
- 2023Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Driving
- 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving
- 2026Probing the Reliability of Driving VLMs: From Inconsistent Responses to Grounded Temporal Reasoning
- 2025From Segments to Scenes: Temporal Understanding for Agentic Autonomous Driving via Vision-Language Models