mlsanityData to 7 Oct 2026

3DSRBench: A Comprehensive 3D Spatial Reasoning Benchmark

arXiv 2412.07825 ↗First posted 10 Dec 2024cs.CVcited by 133

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

3D spatial reasoning is the ability to analyze and interpret the positions, orientations, and spatial relationships of objects within the 3D space. This allows models to develop a comprehensive understanding of the 3D scene, enabling their applicability to a broader range of areas, such as autonomous navigation, robotics, and AR/VR. While large multi-modal models (LMMs) have achieved remarkable progress in a wide range of image and video understanding tasks, their capabilities to perform 3D spatial reasoning on diverse natural images are less studied. In this work we present the first comprehensive 3D spatial reasoning benchmark, 3DSRBench, with 2,772 manually annotated visual question-answer pairs across 12 question types. We conduct robust and thorough evaluation of 3D spatial reasoning abilities by balancing data distribution and adopting a novel FlipEval strategy. To further study the robustness of 3D spatial reasoning w.r.t. camera 3D viewpoints, our 3DSRBench includes two subsets with 3D spatial reasoning questions on paired images with common and uncommon viewpoints. We benchmark a wide range of open-sourced and proprietary LMMs, uncovering their limitations in various aspects of 3D awareness, such as height, orientation, location, and multi-object reasoning, as well as their degraded performance on images from uncommon 6D viewpoints. Our 3DSRBench provide valuable findings and insights about future development of LMMs with strong spatial reasoning abilities. Our project page is available at https://3dsrbench.github.io/.

Key facts. Cited by 133 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 45 of its 57 references that are in the map. First posted on arXiv 10 Dec 2024.

Builds on

45 of 57 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation3 Sep
  2. 2024Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMs11 Apr
  3. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan
  4. 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoning30 Oct

Other references · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2023Segment Anythingcited by 8,158
  8. 2023Visual Instruction Tuningcited by 7,792
  9. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  10. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  11. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  12. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
29 more
  1. 2024GPT-4o System Cardcited by 4,467
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  7. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  8. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  9. 2016Concrete Problems in AI Safetycited by 1,582
  10. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  11. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  12. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  13. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  14. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  15. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  16. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 441
  17. 2023Recognize Anything: A Strong Image Tagging Modelcited by 305
  18. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 175
  19. 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 162
  20. 2022Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wildcited by 159
  21. 2022Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoningcited by 90
  22. 2023ChatPose: Chatting about 3D Human Posecited by 71
  23. 2022ComPhy: Compositional Physical Reasoning of Objects and Events from Videoscited by 61
  24. 2024Efficient Large Multi-modal Models via Visual Context Compressioncited by 46
  25. 2024ViTamin: Designing Scalable Vision Models in the Vision-Language Eracited by 39
  26. 2024ImageNet3D: Towards General-Purpose Object-Level 3D Understandingcited by 25
  27. 2024Compositional 4D Dynamic Scenes Understanding with Physics Priors for Video Question Answeringcited by 18
  28. 20233D-Aware Visual Question Answering about Parts, Poses and Occlusionscited by 16
  29. 2024Generative World Explorercited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 47; 2026: 86.

133 papers.

Sort:

By area:

  1. 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 169
  2. 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3Dcited by 127
  3. 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Modelscited by 110
  4. 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 94
  5. 2025Visual Spatial Tuningcited by 85
  6. 2025MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMscited by 72
  7. 2025SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoningcited by 65
  8. 2025Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Viewscited by 58
  9. 2025LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning?cited by 49
  10. 2025Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulationcited by 41
  11. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 41
  12. 2025Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learningcited by 38
38 more
  1. 20253D Aware Region Prompted Vision Language Modelcited by 31
  2. 2025Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMscited by 30
  3. 2025Holistic Evaluation of Multimodal LLMs on Spatial Intelligencecited by 28
  4. 2026SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecturecited by 27
  5. 2025SITE: towards Spatial Intelligence Thorough Evaluationcited by 26
  6. 2025Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipescited by 22
  7. 2025SpaCE-10: A Comprehensive Benchmark for Multimodal Large Language Models in Compositional Spatial Intelligencecited by 22
  8. 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methodscited by 22
  9. 2026Orient Anything V2: Unifying Orientation and Rotation Understandingcited by 22
  10. 2025DSI-Bench: A Benchmark for Dynamic Spatial Intelligencecited by 20
  11. 2025SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Datacited by 19
  12. 2025SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligencecited by 19
  13. 2026ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loopcited by 19
  14. 2026Theory of Space: Can Foundation Models Construct Spatial Beliefs through Active Exploration?cited by 18
  15. 2026HY-Embodied-0.5: Embodied Foundation Models for Real-World Agentscited by 18
  16. 2026Current Agents Fail to Leverage World Model as Tool for Foresightcited by 15
  17. 2025SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewardscited by 14
  18. 2025FloorplanQA: A Benchmark for Spatial Reasoning in LLMs using Structured Representationscited by 13
  19. 2025SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMscited by 12
  20. 2025SURPRISE3D: A Dataset for Spatial Understanding and Reasoning in Complex 3D Scenescited by 11
  21. 2025SpatialTree: How Spatial Abilities Branch Out in MLLMscited by 11
  22. 2026On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMscited by 11
  23. 2025Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Modelscited by 10
  24. 2025SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagerycited by 10
  25. 2026JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generationcited by 10
  26. 2026MemoBench: Benchmarking World Modeling in Dynamically Changing Environmentscited by 10
  27. 2025GenSpace: Benchmarking Spatially-Aware Image Generationcited by 9
  28. 2025UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic Encodingcited by 9
  29. 2025Can World Models Benefit VLMs for World Dynamics?cited by 9
  30. 2025Mind Meets Space: Rethinking Agentic Spatial Intelligence from a Neuroscience-inspired Perspectivecited by 7
  31. 2026CausalSpatial: A Benchmark for Object-Centric Causal Spatial Reasoningcited by 7
  32. 2026S-Agent: Spatial Tool-Use Elicits Reasoning for Spatial Intelligencecited by 7
  33. 2026Hy-Embodied-VLM-1.0: Efficient Physical-World Agentscited by 7
  34. 2026Chain-of-Thought Degrades Visual Spatial Reasoning Capabilities of Multimodal LLMscited by 6
  35. 2026From Pixels to Words -- Towards Native One-Vision Models at Scalecited by 6
  36. 2026Orca: The World is in Your Mindcited by 6
  37. 2025IndustryNav: Exploring Spatial Reasoning of Embodied Agents in Dynamic Industrial Navigationcited by 5
  38. 2025Vision-Language Memory for Spatial Reasoningcited by 5

50 most cited of 133 All 133 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Spatial457: A Diagnostic Benchmark for 6D Spatial Reasoning of Large Multimodal Models
  2. 2026Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces
  3. 2025Open3D-VQA: A Benchmark for Comprehensive Spatial Reasoning with Multimodal Large Language Model in Open Space
  4. 2024Space3D-Bench: Spatial 3D Question Answering Benchmark
  5. 2026SceneBench: A Hierarchical Benchmark for Vision-Language Understanding of 3D Scenes
  6. 2026Do 3D Large Language Models Really Understand 3D Spatial Relationships?
  7. 2026SpatiaLab: Can Vision-Language Models Perform Spatial Reasoning in the Wild?
  8. 2025SpatialMosaic: A Multiview VLM Dataset for Partial Visibility