mlsanityData to 14 Oct 2026

How Well Do Vision--Language Models Understand Cities? A Comparative Study on Spatial Reasoning from Street-View Images

arXiv 2508.21565 ↗First posted 29 Aug 2025cs.CVcited by 2

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Effectively understanding urban scenes requires fine-grained spatial reasoning about objects, layouts, and depth cues. However, how well current vision-language models (VLMs), pretrained on general scenes, transfer these abilities to urban domain remains underexplored. To address this gap, we conduct a comparative study of three off-the-shelf VLMs-BLIP-2, InstructBLIP, and LLaVA-1.5-evaluating both zero-shot performance and the effects of fine-tuning with a synthetic VQA dataset specific to urban scenes. We construct such dataset from segmentation, depth, and object detection predictions of street-view images, pairing each question with LLM-generated Chain-of-Thought (CoT) answers for step-by-step reasoning supervision. Results show that while VLMs perform reasonably well in zero-shot settings, fine-tuning with our synthetic CoT-supervised dataset substantially boosts performance, especially for challenging question types such as negation and counterfactuals. This study introduces urban spatial reasoning as a new challenge for VLMs and demonstrates synthetic dataset construction as a practical path for adapting general-purpose models to specialized domains.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 52 references that are in the map. First posted on arXiv 29 Aug 2025.

Builds on

28 of 52 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan

Other references · 27

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2020End-to-End Object Detection with Transformerscited by 7,414
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  8. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  10. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  11. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  12. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
15 more
  1. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  2. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  3. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 280
  4. 2020Counterfactual VQA: A Cause-Effect Look at Language Biascited by 266
  5. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 243
  6. 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoningcited by 143
  7. 2024Improve Vision Language Model Chain-of-thought Reasoningcited by 136
  8. 2016Deep Learning the City : Quantifying Urban Perception At A Global Scalecited by 65
  9. 2020VQA-LOL: Visual Question Answering under the Lens of Logiccited by 48
  10. 2024Is Synthetic Data all We Need? Benchmarking the Robustness of Models Trained with Synthetic Imagescited by 41
  11. 2024Efficacy of Synthetic Data as a Benchmarkcited by 30
  12. 2025NegVQA: Can Vision Language Models Understand Negation?cited by 13
  13. 2023A New Benchmark: On the Utility of Synthetic Data with Blender for Bare Supervised Learning and Downstream Domain Adaptationcited by 11
  14. 2025Negation: A Pink Elephant in the Large Language Models' Room?cited by 7
  15. 2025Generalizing vision-language models to novel domains: A comprehensive surveycited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learningcited by 41
  2. 2026SpatialASM: Internalizing Active Mentalization for Spatial Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data
  2. 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models
  3. 2026SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models
  4. 2024Reasoning Paths with Reference Objects Elicit Quantitative Spatial Reasoning in Large Vision-Language Models
  5. 2026USAI-Quant: A Quantitative Reasoning Benchmark for Vision-Language Models in Built Environments
  6. 2025Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas
  7. 2026m2sv: A Scalable Benchmark for Map-to-Street-View Spatial Reasoning
  8. 2026MindEdit-Bench: Benchmarking Object-Level Counterfactual Spatial Reasoning in VLMs from In-the-Wild Photos