CityToolVQA: Tool-Augmented Visual Question Answering for 3D Spatial Cognition in Urban Low-Altitude Environments
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models also fits: VLM benchmarks and visual reasoning
Abstract (arXiv)
CityToolVQA addresses the weak performance of Vision-Language Models (VLMs) on quantitative tasks in urban low-altitude visual question answering. We divide the seven tasks into qualitative and quantitative groups: qualitative questions are answered directly by the VLM, whereas quantitative questions are handled by an external visual-geometric toolchain that performs object grounding, segmentation, depth back-projection, and spatial computation. The toolchain can be attached to different VLMs in a zero-shot manner; a Depth-Assisted Prompt Inference (DAPI) fallback is triggered when the main-chain detection is invalid or unreliable, and CityToolVQA-SFT adapts the 8B backbone to tool-conditioned inputs. On the 73,324-question Open3D-VQA-v2 test set, CityToolVQA-SFT (Qwen3-VL-8B) reaches 67.6% overall accuracy, and attaching the toolchain to ten open-source VLMs improves quantitative-task accuracy by 12.7-36.9 percentage points. These results indicate that externalizing explicit 3D geometric computation effectively complements the limited ability of RGB-only VLMs to estimate metric distances and object sizes.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 26 references that are in the map. First posted on arXiv 26 Sep 2026.
Builds on
21 of 26 references.
Closest ancestors · 4
Same direction, newest first
- 2026DM-KG: A Novel Method for Boosting Spatial Cognition of Vision-Language Models in Street View Imagery14 Jul
- 2025SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RL3 Dec
- 2025Geometrically-Constrained Agent for Spatial Reasoning27 Nov
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan
Other references · 17
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
- 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
- 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
- 2022Visual Programming: Compositional visual reasoning without trainingcited by 507
- 2025Kimi-VL Technical Reportcited by 381
- 2023Segment Anything in High Qualitycited by 373
5 more
- 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
- 2025CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Spacecited by 31
- 2025ToolVQA: A Dataset for Multi-step Reasoning VQA with External Toolscited by 13
- 2026S-Agent: Spatial Tool-Use Elicits Reasoning for Spatial Intelligencecited by 7
- 2026MASER: Modality-Adaptive Specialist Routing for Embodied 3D Spatial Intelligencecited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Inter-3D VQA: A Roadside Multimodal Benchmark for 3D Spatiotemporally Grounded Visual Question Answering
- 2026SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models
- 2024SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priors
- 2026SpatialSpeak: QA-Native Reconstruction with Local and Global Context for Spatial Chain-of-Thought Reasoning
- 2026SpatialChain: A Benchmark for Auditing Spatial Reasoning Faithfulness in VLMs
- 2025SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoning
- 2026SPATIOROUTE: Dynamic Prompt Routing for Zero-Shot Spatial Reasoning
- 2025How Well Do Vision--Language Models Understand Cities? A Comparative Study on Spatial Reasoning from Street-View Images