mlsanityData to 7 Oct 2026

Map2Thought: Explicit 3D Spatial Reasoning via Metric Cognitive Maps

arXiv 2601.11442 ↗First posted 16 Jan 2026cs.CV cs.AIcited by 8

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

We propose Map2Thought, a framework that enables explicit and interpretable spatial reasoning for 3D VLMs. The framework is grounded in two key components: Metric Cognitive Map (Metric-CogMap) and Cognitive Chain-of-Thought (Cog-CoT). Metric-CogMap provides a unified spatial representation by integrating a discrete grid for relational reasoning with a continuous, metric-scale representation for precise geometric understanding. Building upon the Metric-CogMap, Cog-CoT performs explicit geometric reasoning through deterministic operations, including vector operations, bounding-box distances, and occlusion-aware appearance order cues, producing interpretable inference traces grounded in 3D structure. Experimental results show that Map2Thought enables explainable 3D understanding, achieving 59.9% accuracy using only half the supervision, closely matching the 60.9% baseline trained with the full dataset. It consistently outperforms state-of-the-art methods by 5.3%, 4.8%, and 4.0% under 10%, 25%, and 50% training subsets, respectively, on the VSI-Bench.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 49 of its 54 references that are in the map. First posted on arXiv 16 Jan 2026.

Builds on

49 of 54 references.

Closest ancestors · 3

Same direction, newest first

  1. 20253DRS: MLLMs Need 3D-Aware Representation Supervision for Scene Understanding2 Jun
  2. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
  3. 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning30 Nov

Other references · 46

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  4. 2025Qwen2.5-VL Technical Reportcited by 5,418
  5. 2024GPT-4o System Cardcited by 4,467
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  8. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  9. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  10. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  11. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  12. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
34 more
  1. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  2. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  3. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
  4. 2023VILA: On Pre-training for Visual Language Modelscited by 686
  5. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
  6. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  7. 2024Long Context Transfer from Language to Visioncited by 502
  8. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 446
  9. 2023An Embodied Generalist Agent in 3D Worldcited by 355
  10. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
  11. 2020Scan2Cap: Context-aware Dense Captioning in RGB-D Scanscited by 215
  12. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 206
  13. 2024LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awarenesscited by 178
  14. 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstructioncited by 154
  15. 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understandingcited by 131
  16. 2023Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenescited by 125
  17. 2022Language Conditioned Spatial Relation Reasoning for 3D Object Groundingcited by 121
  18. 2025Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geometry Priorscited by 111
  19. 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Modelscited by 106
  20. 2025MindCube: Spatial Mental Modeling from Limited Viewscited by 91
  21. 2022UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Groundingcited by 69
  22. 20253D-LLaVA: Towards Generalist 3D LMMs with Omni Superpoint Transformercited by 67
  23. 2025Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awarenesscited by 61
  24. 2021D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Groundingcited by 52
  25. 2025Inst3D-LMM: Instance-Aware 3D Scene Understanding with Multi-modal Instruction Tuningcited by 50
  26. 20213D Question Answeringcited by 41
  27. 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 41
  28. 20243DMIT: 3D Multi-modal Instruction Tuning for Scene Understandingcited by 32
  29. 2024CogNav: Cognitive Process Modeling for Object Goal Navigation with LLMscited by 32
  30. 2025Spatial Understanding from Videos: Structured Prompts Meet Simulation Datacited by 28
  31. 2025CoMapGS: Covisibility Map-based Gaussian Splatting for Sparse Novel View Synthesiscited by 10
  32. 2025UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic Encodingcited by 9
  33. 2025Does Your 3D Encoder Really Work? When Pretrain-SFT from 2D VLMs Meets 3D VLMscited by 8
  34. 2023Generating Context-Aware Natural Answers for Questions in 3D Scenescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026Communicating about Space: Language-Mediated Spatial Integration Across Partial Viewscited by 2
  2. 2026GAP-MLLM: Geometry-Aligned Pre-training for Activating 3D Spatial Perception in Multimodal Large Language Modelscited by 1
  3. 2026SpaceMind++: Toward Allocentric Cognitive Maps for Spatially Grounded Video MLLMscited by 1
  4. 2026Bridging the Detection-to-Abstention Gap in Reasoning Models under Insufficient Informationcited by 1
  5. 2026Ouroboros-Spatial: Closing the Data-Model Loop for Spatial Reasoningcited by 1
  6. 2026ConsiSpace: Learning Geometric Consistency Matters for Video Spatial Reasoningcited by 1
  7. 2026SpatialSpeak: QA-Native Reconstruction with Local and Global Context for Spatial Chain-of-Thought Reasoningcited by 0
  8. 2026From Reasoning Failures to Composable Video Spatial Intelligencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Cog3DMap: Multi-View Vision-Language Reasoning with 3D Cognitive Maps
  2. 2025Video2Layout: Recall and Reconstruct Metric-Grounded Cognitive Map for Spatial Reasoning
  3. 2025Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views
  4. 2025DynaSolidGeo: A Dynamic Benchmark for Genuine Spatial Mathematical Reasoning of VLMs in Solid Geometry
  5. 2026World2Mind: Cognition Toolkit for Allocentric Spatial Reasoning in Foundation Models
  6. 2026Disentangling 3D Modeling from Spatial Reasoning
  7. 2026Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces
  8. 2026Boosting MLLM Spatial Reasoning with Geometrically Referenced 3D Scene Representations