mlsanityData to 14 Oct 2026

Open3D-VQA: A Benchmark for Comprehensive Spatial Reasoning with Multimodal Large Language Model in Open Space

arXiv 2503.11094 ↗First posted 14 Mar 2025cs.CVcited by 39

On the map: LLM reasoning → Spatial and embodied reasoning

also in 1 more
Abstract (arXiv)

Spatial reasoning is a fundamental capability of multimodal large language models (MLLMs), yet their performance in open aerial environments remains underexplored. In this work, we present Open3D-VQA, a novel benchmark for evaluating MLLMs' ability to reason about complex spatial relationships from an aerial perspective. The benchmark comprises 73k QA pairs spanning 7 general spatial reasoning tasks, including multiple-choice, true/false, and short-answer formats, and supports both visual and point cloud modalities. The questions are automatically generated from spatial relations extracted from both real-world and simulated aerial scenes. Evaluation on 13 popular MLLMs reveals that: 1) Models are generally better at answering questions about relative spatial relations than absolute distances, 2) 3D LLMs fail to demonstrate significant advantages over 2D LLMs, and 3) Fine-tuning solely on the simulated dataset can significantly improve the model's spatial reasoning performance in real-world scenarios. We release our benchmark, data generation pipeline, and evaluation toolkit to support further research: https://github.com/EmbodiedCity/Open3D-VQA.code.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 23 of its 53 references that are in the map. First posted on arXiv 14 Mar 2025.

Builds on

23 of 53 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
  2. 2024SURDS: Benchmarking Spatial Understanding and Reasoning in Driving Scenarios with Vision Language Models20 Nov
  3. 2024An Empirical Analysis on Spatial Reasoning Capabilities of Large Multimodal Models9 Nov
  4. 2024Multi-modal Situated Reasoning in 3D Scenes4 Sep
  5. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation3 Sep
  6. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models3 Jun
  7. 2022SQA3D: Situated Question Answering in 3D Scenes14 Oct

Other references · 16

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2024GPT-4o System Cardcited by 4,588
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  7. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  8. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
  9. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
  10. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
  11. 2023An Embodied Generalist Agent in 3D Worldcited by 365
  12. 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Modelscited by 107
4 more
  1. 2023KOSMOS-2.5: A Multimodal Literate Modelcited by 96
  2. 2024Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understandingcited by 49
  3. 2024EmbodiedCity: A Benchmark Platform for Embodied Agent in Real-world City Environmentcited by 42
  4. 2025UrbanVideo-Bench: Benchmarking Vision-Language Models on Embodied Intelligence with Video Data in Urban Spacescited by 36

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 16; 2026: 23.

39 papers.

Sort:

By area:

  1. 2025WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agentcited by 119
  2. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 90
  3. 2025Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Viewscited by 65
  4. 2025MetaSpatial: Reinforcing 3D Spatial Reasoning in VLMs for the Metaversecited by 51
  5. 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methodscited by 23
  6. 2025From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMscited by 12
  7. 2025SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagerycited by 11
  8. 2026CityCube: Benchmarking Cross-view Spatial Reasoning on Vision-Language Models in Urban Environmentscited by 11
  9. 2025Dissecting Embodied Abilities in Multimodal Language Models through Skill-level Evaluation and Diagnosiscited by 8
  10. 2026STReasoner: Empowering LLMs for Spatio-Temporal Reasoning in Time Series via Spatial-Aware Reinforcement Learningcited by 7
  11. 2026SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Modelscited by 6
  12. 2026SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environmentscited by 6
27 more
  1. 2026OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMscited by 6
  2. 2025USTBench: Benchmarking and Dissecting Spatiotemporal Reasoning of LLMs as Urban Agentscited by 5
  3. 2025IndustryNav: Exploring Spatial Reasoning of Embodied Agents in Dynamic Industrial Navigationcited by 5
  4. 2025ReasonX: MLLM-Guided Intrinsic Image Decompositioncited by 5
  5. 2026EarthSpatialBench: Benchmarking Spatial Reasoning Capabilities of Multimodal LLMs on Earth Imagerycited by 5
  6. 2025Benchmarking Spatiotemporal Reasoning in LLMs and Reasoning Models: Capabilities and Challengescited by 4
  7. 2026VRIQ: Benchmarking and Analyzing Visual-Reasoning IQ of VLMscited by 4
  8. 2025Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AIcited by 3
  9. 2025Reasoning Path and Latent State Analysis for Multi-view Visual Spatial Reasoning: A Cognitive Science Perspectivecited by 3
  10. 2026ESPIRE: A Diagnostic Benchmark for Embodied Spatial Reasoning of Vision-Language Modelscited by 3
  11. 2025MPCC: A Novel Benchmark for Multimodal Planning with Complex Constraints in Multimodal Large Language Modelscited by 2
  12. 2026How Far Are Vision-Language Models from Constructing the Real World? A Benchmark for Physical Generative Reasoningcited by 2
  13. 2026ArchSIBench: Benchmarking the Architectural Spatial Intelligence of Vision-Language Modelscited by 2
  14. 2026SpatialUAV: Benchmarking Spatial Intelligence for Low-Altitude UAV Perception, Collaboration, and Motioncited by 2
  15. 20263DCity-LLM: Empowering Multi-modality Large Language Models for 3D City-scale Perception and Understandingcited by 1
  16. 2026Learning Multi-View Spatial Reasoning from Cross-View Relationscited by 1
  17. 2026GeoAnchor: Collaborative Reasoning via Latent Decomposition for 3D Spatial Understandingcited by 1
  18. 2026CosFly-VLA: A Spatially Aware Vision-Language-Action Model for UAV Trackingcited by 1
  19. 2025Ascending the Infinite Ladder: Benchmarking Spatial Deformation Reasoning in Vision-Language Modelscited by 0
  20. 2026CoMa: Contextual Massing Generation with Vision-Language Modelscited by 0
  21. 2026From Perception to Action: Spatial AI Agents and World Modelscited by 0
  22. 2026How Far Are Large Multimodal Models from Human-Level Spatial Action? A Benchmark for Goal-Oriented Embodied Navigation in Urban Airspacecited by 0
  23. 2026PinpointQA: A Benchmark for Small Object-Centric Spatial Understanding in Indoor Videoscited by 0
  24. 2026Scaling Diverse Language Generation for 3D Visual Groundingcited by 0
  25. 2026Inter-3D VQA: A Roadside Multimodal Benchmark for 3D Spatiotemporally Grounded Visual Question Answeringcited by 0
  26. 2026AirAnchor: Bridging Local and Global Spatial Information for Zero-Shot Aerial Vision-and-Language Navigationcited by 0
  27. 2026Towards a Data-Centric Understanding of Spatial Reasoning in Multimodal Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025The Point, the Vision and the Text: Does Point Cloud Boost Spatial Reasoning of Large Language Models? A Bias-Controlled Study
  2. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities
  3. 2025Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipes
  4. 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence
  5. 2026Do 3D Large Language Models Really Understand 3D Spatial Relationships?
  6. 2025SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data
  7. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks
  8. 2025SpatialMosaic: A Multiview VLM Dataset for Partial Visibility