mlsanityData to 7 Oct 2026

CityToolVQA: Tool-Augmented Visual Question Answering for 3D Spatial Cognition in Urban Low-Altitude Environments

arXiv 2609.32427 ↗First posted 26 Sep 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models also fits: VLM benchmarks and visual reasoning

Abstract (arXiv)

CityToolVQA addresses the weak performance of Vision-Language Models (VLMs) on quantitative tasks in urban low-altitude visual question answering. We divide the seven tasks into qualitative and quantitative groups: qualitative questions are answered directly by the VLM, whereas quantitative questions are handled by an external visual-geometric toolchain that performs object grounding, segmentation, depth back-projection, and spatial computation. The toolchain can be attached to different VLMs in a zero-shot manner; a Depth-Assisted Prompt Inference (DAPI) fallback is triggered when the main-chain detection is invalid or unreliable, and CityToolVQA-SFT adapts the 8B backbone to tool-conditioned inputs. On the 73,324-question Open3D-VQA-v2 test set, CityToolVQA-SFT (Qwen3-VL-8B) reaches 67.6% overall accuracy, and attaching the toolchain to ten open-source VLMs improves quantitative-task accuracy by 12.7-36.9 percentage points. These results indicate that externalizing explicit 3D geometric computation effectively complements the limited ability of RGB-only VLMs to estimate metric distances and object sizes.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 26 references that are in the map. First posted on arXiv 26 Sep 2026.

Builds on

21 of 26 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026DM-KG: A Novel Method for Boosting Spatial Cognition of Vision-Language Models in Street View Imagery14 Jul
  2. 2025SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RL3 Dec
  3. 2025Geometrically-Constrained Agent for Spatial Reasoning27 Nov
  4. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan

Other references · 17

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  3. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  4. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  5. 2025Qwen3-VL Technical Reportcited by 2,534
  6. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  7. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  8. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
  9. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
  10. 2022Visual Programming: Compositional visual reasoning without trainingcited by 507
  11. 2025Kimi-VL Technical Reportcited by 381
  12. 2023Segment Anything in High Qualitycited by 373
5 more
  1. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
  2. 2025CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Spacecited by 31
  3. 2025ToolVQA: A Dataset for Multi-step Reasoning VQA with External Toolscited by 13
  4. 2026S-Agent: Spatial Tool-Use Elicits Reasoning for Spatial Intelligencecited by 7
  5. 2026MASER: Modality-Adaptive Specialist Routing for Embodied 3D Spatial Intelligencecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Inter-3D VQA: A Roadside Multimodal Benchmark for 3D Spatiotemporally Grounded Visual Question Answering
  2. 2026SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models
  3. 2024SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priors
  4. 2026SpatialSpeak: QA-Native Reconstruction with Local and Global Context for Spatial Chain-of-Thought Reasoning
  5. 2026SpatialChain: A Benchmark for Auditing Spatial Reasoning Faithfulness in VLMs
  6. 2025SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoning
  7. 2026SPATIOROUTE: Dynamic Prompt Routing for Zero-Shot Spatial Reasoning
  8. 2025How Well Do Vision--Language Models Understand Cities? A Comparative Study on Spatial Reasoning from Street-View Images