mlsanityData to 14 Oct 2026

SpatialForge: Bootstrapping 3D-Aware Spatial Reasoning from Open-World 2D Images

arXiv 2605.11462 ↗First posted 12 May 2026cs.CV cs.AIcited by 0

On the map: LLM reasoning → Spatial and embodied reasoning

also in 2 more
Abstract (arXiv)

Recent advancements in Large Vision-Language Models (VLMs) have demonstrated exceptional semantic understanding, yet these models consistently struggle with spatial reasoning, often failing at fundamental geometric tasks such as depth ordering and precise coordinate grounding. Recent efforts introduce spatial supervision from scene-centric datasets (e.g., multi-view scans or indoor video), but are constrained by the limited number of underlying scenes. As a result, the scale and diversity of such data remain significantly smaller than those of web-scale 2D image collections. To address this limitation, we propose SpatialForge, a scalable data synthesis pipeline that transforms in-the-wild 2D images into spatial reasoning supervision. Our approach decomposes spatial reasoning into perception and relation, and constructs structured supervision signals covering depth, layout, and viewpoint-dependent reasoning, with automatic verification to ensure data quality. Based on this pipeline, we build SpatialForge-10M, a large-scale dataset containing 10 million spatial QA pairs. Extensive experiments across multiple spatial reasoning benchmarks demonstrate that training on SpatialForge-10M significantly improves the spatial reasoning ability of standard VLMs, highlighting the effectiveness of scaling 2D data for 3D-aware spatial reasoning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 45 of its 52 references that are in the map. First posted on arXiv 12 May 2026.

Builds on

45 of 52 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026LAST: Leveraging Tools as Hints to Enhance Spatial Reasoning for Multimodal Large Language Models8 Apr
  2. 2025SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Models9 Oct
  3. 2025InternSpatial: A Comprehensive Dataset for Spatial Reasoning in Vision-Language Models23 Jun
  4. 2025OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models3 Jun
  5. 2025SpaceR: Reinforcing MLLMs in Video Spatial Reasoning2 Apr
  6. 2025SpatialCoT: Advancing Spatial Reasoning through Coordinate Alignment and Chain-of-Thought for Embodied Task Planning17 Jan
  7. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
  8. 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models21 Jun
  9. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models3 Jun
  10. 2024Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMs11 Apr
  11. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan

Other references · 34

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  7. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  8. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  9. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  10. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  11. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  12. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
22 more
  1. 2024Depth Anything V2cited by 1,422
  2. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  3. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  4. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  5. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
  6. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
  7. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 603
  8. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
  9. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 333
  10. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 329
  11. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 217
  12. 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 216
  13. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 206
  14. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 173
  15. 2024SpatialBot: Precise Spatial Understanding with Vision Language Modelscited by 152
  16. 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3Dcited by 138
  17. 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interactioncited by 137
  18. 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Roboticscited by 136
  19. 2024HourVideo: 1-Hour Video-Language Understandingcited by 113
  20. 2024Ferret-v2: An Improved Baseline for Referring and Grounding with Large Language Modelscited by 97
  21. 2025SpaCE-10: A Comprehensive Benchmark for Multimodal Large Language Models in Compositional Spatial Intelligencecited by 23
  22. 2024Proximity QA: Unleashing the Power of Multi-Modal Large Language Models for Spatial Proximity Analysiscited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Spatial Reasoning in Foundation Models: Benchmarking Object-Centric Spatial Understanding
  2. 2025Spatial-ViLT: Enhancing Visual Spatial Reasoning through Multi-Task Learning
  3. 2025Spatial Understanding from Videos: Structured Prompts Meet Simulation Data
  4. 2025SpatialMosaic: A Multiview VLM Dataset for Partial Visibility
  5. 2026Make Geometry Matter for Spatial Reasoning
  6. 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmark
  7. 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models
  8. 2025SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards