mlsanityData to 14 Oct 2026

Are VLMs Ready for Lane Topology Awareness in Autonomous Driving?

arXiv 2509.16654 ↗First posted 20 Sep 2025cs.CVcited by 0

On the map: LLM reasoning → Chain-of-thought reasoning for driving

also in 2 more
Abstract (arXiv)

Vision-Language Models (VLMs) have recently shown remarkable progress in multimodal reasoning, yet their applications in autonomous driving remain limited. In particular, the ability to understand road topology, a key requirement for safe navigation, has received relatively little attention. While some recent works have begun to explore VLMs in driving contexts, their performance on topology reasoning is far from satisfactory. In this work, we systematically evaluate VLMs' capabilities in road topology understanding. Specifically, multi-view images are projected into unified ground-plane coordinate system and fused into bird's-eye-view (BEV) lanes. Based on these BEV lanes, we formulate four topology-related diagnostic VQA tasks, which together capture essential components of spatial topology reasoning. Through extensive evaluation, we find that while frontier closed-source models (e.g., GPT-4o) achieve relatively high accuracy in some tasks, they still fail in some spatial questions that humans can answer (e.g., GPT-4o achieve only 67.8% in vector, a two-class classification problem). Furthermore, we find open-source VLMs, even at 30B scale, struggle significantly. These results indicate that spatial reasoning remains a fundamental bottleneck for current VLMs. We also find that the model's capability is positively correlated with model size, length of reasoning tokens and shots provided as examples, showing direction for future research.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning. Builds on 25 of its 28 references that are in the map. First posted on arXiv 20 Sep 2025.

Builds on

25 of 28 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning6 Apr

Other references · 24

  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2024GPT-4o System Cardcited by 4,588
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  6. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  7. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  8. 2025Gemma 3 Technical Reportcited by 1,824
  9. 2025s1: Simple test-time scalingcited by 1,341
  10. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  11. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
  12. 2025Kimi-VL Technical Reportcited by 388
12 more
  1. 2025SmolVLM: Redefining small and efficient multimodal modelscited by 237
  2. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  3. 2024Pixtral 12Bcited by 179
  4. 2023Dolphins: Multimodal Language Model for Drivingcited by 124
  5. 2025Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectivescited by 122
  6. 2025MiMo-VL Technical Reportcited by 115
  7. 2023Augmenting Lane Perception and Topology Understanding with Standard Definition Navigation Mapscited by 58
  8. 2024RoboTron-Drive: All-in-One Large Multimodal Model for Autonomous Drivingcited by 30
  9. 2024TopoLogic: An Interpretable Pipeline for Lane Topology Reasoning on Driving Scenescited by 29
  10. 2025Chameleon: Fast-slow Neuro-symbolic Lane Topology Extractioncited by 16
  11. 2025Trajectory Prediction Meets Large Language Models: A Surveycited by 14
  12. 2025TopoPoint: Enhance Topology Reasoning via Endpoint Detection in Autonomous Drivingcited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025From Static to Dynamic: a Survey of Topology-Aware Perception in Autonomous Driving
  2. 2025Geometry-Guided Representations for Coherent Lane and Traffic Topology Reasoning in Driving Scenes
  3. 2025VLM-3D:End-to-End Vision-Language Models for Open-World 3D Perception
  4. 2025NuScenes-SpatialQA: A Spatial Understanding and Reasoning Benchmark for Vision-Language Models in Autonomous Driving
  5. 2025FASTopoWM: Fast-Slow Lane Segment Topology Reasoning with Latent World Models
  6. 2025SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving
  7. 2026GeoDrive-Bench: Benchmarking Region-Specific Multimodal Reasoning in Autonomous Driving
  8. 2026From Steering to Pedalling: Do Autonomous Driving VLMs Generalize to Cyclist-Assistive Spatial Perception and Planning?