mlsanityData to 14 Oct 2026

SD-VLM: Spatial Measuring and Understanding with Depth-Encoded Vision-Language Models

arXiv 2509.17664 ↗First posted 22 Sep 2025cs.CV cs.AIcited by 25

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

While vision language models (VLMs) excel in 2D semantic visual understanding, their ability to quantitatively reason about 3D spatial relationships remains under-explored, due to the deficiency of 2D images' spatial representation ability. In this paper, we analyze the problem hindering VLMs' spatial understanding abilities and propose SD-VLM, a novel framework that significantly enhances fundamental spatial perception abilities of VLMs through two key contributions: (1) propose Massive Spatial Measuring and Understanding (MSMU) dataset with precise spatial annotations, and (2) introduce a simple depth positional encoding method strengthening VLMs' spatial awareness. MSMU dataset covers massive quantitative spatial tasks with 700K QA pairs, 2.5M physical numerical annotations, and 10K chain-of-thought augmented samples. We have trained SD-VLM, a strong generalist VLM which shows superior quantitative spatial measuring and understanding capability. SD-VLM not only achieves state-of-the-art performance on our proposed MSMU-Bench, but also shows spatial generalization abilities on other spatial understanding benchmarks including Q-Spatial and SpatialRGPT-Bench. Extensive experiments demonstrate that SD-VLM outperforms GPT-4o and Intern-VL3-78B by 26.91% and 25.56% respectively on MSMU-Bench. Code and models are released at https://github.com/cpystan/SD-VLM.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 51 of its 64 references that are in the map. First posted on arXiv 22 Sep 2025.

Builds on

51 of 64 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMs17 Mar
  2. 2025iVISPAR -- An Interactive Visual-Spatial Reasoning Benchmark for VLMs5 Feb
  3. 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models2 Jan
  4. 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics25 Nov
  5. 2024An Empirical Analysis on Spatial Reasoning Capabilities of Large Multimodal Models9 Nov
  6. 2024Do Vision-Language Models Represent Space and How? Evaluating Spatial Frame of Reference Under Ambiguities22 Oct
  7. 2024Reasoning Paths with Reference Objects Elicit Quantitative Spatial Reasoning in Large Vision-Language Models15 Sep
  8. 2024SpatialBot: Precise Spatial Understanding with Vision Language Models19 Jun
  9. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models3 Jun
  10. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs12 Feb
  11. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan
  12. 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning30 Nov
2 more
  1. 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoning30 Oct
  2. 2023Physically Grounded Vision-Language Models for Robotic Manipulation5 Sep

Other references · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2024Qwen2.5 Technical Reportcited by 4,942
  7. 2024GPT-4o System Cardcited by 4,588
  8. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  9. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  10. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  11. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  12. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
25 more
  1. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  2. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  3. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  4. 2019Towards VQA Models That Can Readcited by 1,718
  5. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  6. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
  7. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
  8. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  9. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
  10. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  11. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
  12. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 382
  13. 2023An Embodied Generalist Agent in 3D Worldcited by 365
  14. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
  15. 2022SQA3D: Situated Question Answering in 3D Scenescited by 314
  16. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 305
  17. 2022Visual Spatial Reasoningcited by 293
  18. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 146
  19. 2024Does Spatial Cognition Emerge in Frontier Models?cited by 66
  20. 2019SpatialSense: An Adversarially Crowdsourced Benchmark for Spatial Relation Recognitioncited by 55
  21. 2024MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language Annotationscited by 40
  22. 2023Tame a Wild Camera: In-the-Wild Monocular Camera Calibrationcited by 34
  23. 2024Lost in Space: Probing Fine-grained Spatial Understanding in Vision and Language Resamplerscited by 11
  24. 2024VR-GPT: Visual Language Model for Intelligent Virtual Reality Applicationscited by 9
  25. 2024Analyzing Multimodal Interaction Strategies for LLM-Assisted Manipulation of 3D Scenescited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 21.

25 papers.

Sort:

By area:

  1. 2026RHO: Your Coding Agent is Secretly a Roboticistcited by 20
  2. 2025Towards Spatial Trace with Reasoning in Vision-Language Models for Roboticscited by 12
  3. 2025SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagerycited by 11
  4. 2026Allocentric Perceiver: Disentangling Allocentric Reasoning from Egocentric Visual Priors via Frame Instantiationcited by 5
  5. 2025Seeing through Imagination: Learning Scene Geometry via Implicit Spatial World Modelingcited by 3
  6. 2026From Correspondence to Actions: Human-Like Multi-Image Spatial Reasoning in Multi-modal Large Language Modelscited by 3
  7. 2026OmniLottie: Generating Vector Animations via Parameterized Lottie Tokenscited by 3
  8. 2026HiSpatial: Taming Hierarchical 3D Spatial Understanding in Vision-Language Modelscited by 3
  9. 2026LAST: Leveraging Tools as Hints to Enhance Spatial Reasoning for Multimodal Large Language Modelscited by 3
  10. 2026Robo-Harness K1: Harnessing Robot-Use Agents via Perception Augmentationcited by 3
  11. 20254D-RGPT: Toward Region-level 4D Understanding via Perceptual Distillationcited by 2
  12. 2026AlloSpatial: Agentic Harness Framework for Spatial Reasoning in Foundation Modelscited by 2
13 more
  1. 2026World2Mind: Cognition Toolkit for Allocentric Spatial Reasoning in Foundation Modelscited by 1
  2. 2026ESC: Emotional Self-Correction for Reliable Vision-Language Modelscited by 1
  3. 2026PhysQuantAgent: An Inference Pipeline of Mass Estimation for Vision-Language Modelscited by 0
  4. 2026OneCanvas: 3D Scene Understanding via Panoramic Reprojectioncited by 0
  5. 2026Ground3D-LMM: Fine-Grained 3D Point Grounding and Spatial Reasoning with LMMcited by 0
  6. 2026Traj-VLN: Learning Pixel-Space Interaction via Autoregressive Trajectory Generationcited by 0
  7. 2026When Depth Is Better Told Than Shown: Depth-Ordinal Prompting for Vision-Language Spatial Reasoningcited by 0
  8. 2026ChainSpace: A Chained-Reasoning Paradigm for Spatial Intelligencecited by 0
  9. 2026EmbodiedMind: Adaptive Data Curation and Prefix-Tree Reinforcement Learning for Efficient Embodied Intelligencecited by 0
  10. 2026GaugeVLM: Structuring Spatial Supervision with Measured Geometric Interventionscited by 0
  11. 2026SpatialCORE: Confidence-Aware Grounded Spatial Reasoning in Large Vision--Language Modelscited by 0
  12. 2026Soft Spatial Reasoningcited by 0
  13. 2026GeoReader: Learning Latent Geometry for Spatial Reasoning in Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025G2VLM: Geometry Grounded Vision Language Model with Unified 3D Reconstruction and Spatial Reasoning
  2. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence
  3. 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D
  4. 2026The Spatial Blindspot of Vision-Language Models
  5. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective
  6. 2025Spatial 3D-LLM: Exploring Spatial Awareness in 3D Vision-Language Models
  7. 2025SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data
  8. 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models