mlsanityData to 7 Oct 2026

How to Enable LLM with 3D Capacity? A Survey of Spatial Reasoning in LLM

arXiv 2504.05786 ↗First posted 8 Apr 2025cs.CV cs.AIcited by 41

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

3D spatial understanding is essential in real-world applications such as robotics, autonomous vehicles, virtual reality, and medical imaging. Recently, Large Language Models (LLMs), having demonstrated remarkable success across various domains, have been leveraged to enhance 3D understanding tasks, showing potential to surpass traditional computer vision methods. In this survey, we present a comprehensive review of methods integrating LLMs with 3D spatial understanding. We propose a taxonomy that categorizes existing methods into three branches: image-based methods deriving 3D understanding from 2D visual data, point cloud-based methods working directly with 3D representations, and hybrid modality-based methods combining multiple data streams. We systematically review representative methods along these categories, covering data representations, architectural modifications, and training strategies that bridge textual and 3D modalities. Finally, we discuss current limitations, including dataset scarcity and computational challenges, while highlighting promising research directions in spatial perception, multi-modal fusion, and real-world applications.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 19 of its 46 references that are in the map. First posted on arXiv 8 Apr 2025.

Builds on

19 of 46 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priors18 Mar
  2. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoning18 Mar

Other references · 17

  1. 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 231
  2. 2023Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Followingcited by 192
  3. 2024LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awarenesscited by 178
  4. 2024M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Modelscited by 154
  5. 2023Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenescited by 125
  6. 2024Grounded 3D-LLM with Referent Tokenscited by 111
  7. 2023LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understandingcited by 95
  8. 20253D-LLaVA: Towards Generalist 3D LMMs with Omni Superpoint Transformercited by 67
  9. 2024EmbodiedCity: A Benchmark Platform for Embodied Agent in Real-world City Environmentcited by 42
  10. 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 41
  11. 2024Uni3D-LLM: Unifying Point Cloud Perception, Generation and Editing with Large Language Modelscited by 24
  12. 20243D-CT-GPT: Generating 3D Radiology Reports through Integration of Large Vision-Language Modelscited by 15
5 more
  1. 2024Kestrel: 3D Multimodal LLM for Part-Aware Grounded Descriptioncited by 9
  2. 2024More Text, Less Point: Towards 3D Data-Efficient Point-Language Understandingcited by 9
  3. 2024Argus: Benchmarking and Enhancing Vision-Language Models for 3D Radiology Report Generationcited by 8
  4. 2024LLMI3D: MLLM-based 3D Perception from a Single 2D Imagecited by 7
  5. 2024OpenMEDLab: An Open-source Platform for Multi-modality Foundation Models in Medicinecited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 25; 2026: 16.

41 papers.

Sort:

By area:

  1. 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Modelscited by 110
  2. 2025Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Viewscited by 58
  3. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 41
  4. 2025Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learningcited by 38
  5. 2025D3: Training-Free AI-Generated Video Detection Using Second-Order Featurescited by 32
  6. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 31
  7. 2025Code2Video: A Code-centric Paradigm for Educational Video Generationcited by 23
  8. 2025Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimizationcited by 20
  9. 2025Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenescited by 19
  10. 2025AirCopBench: A Benchmark for Multi-drone Collaborative Embodied Perception and Reasoningcited by 19
  11. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 13
  12. 2025SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagerycited by 10
29 more
  1. 2025AirScape: An Aerial Generative World Model with Motion Controllabilitycited by 7
  2. 2025AInsteinBench: Benchmarking Coding Agents on Scientific Repositoriescited by 6
  3. 2025VS-Bench: Evaluating VLMs for Strategic Abilities in Multi-Agent Environmentscited by 5
  4. 2025S2-MLLM: Boosting Spatial Reasoning Capability of MLLMs for 3D Visual Grounding with Structural Guidancecited by 5
  5. 2026PanoWorld: Towards Spatial Supersensing in 360^circ Panorama Worldcited by 5
  6. 2025Uni3D-MoE: Scalable Multimodal 3D Scene Understanding via Mixture of Expertscited by 4
  7. 2025Descrip3D: Enhancing Large Language Model-based 3D Scene Understanding with Object-Level Text Descriptionscited by 3
  8. 2025MonoSR: Open-Vocabulary Spatial Reasoning from Monocular Imagescited by 3
  9. 2025Seeing through Imagination: Learning Scene Geometry via Implicit Spatial World Modelingcited by 3
  10. 2026Do MLLMs Really Understand Space? A Mathematical Reasoning Evaluationcited by 3
  11. 2026ViSRA: A Video-based Spatial Reasoning Agent for Multi-modal Large Language Modelscited by 3
  12. 2025SNOW: Spatio-Temporal Scene Understanding with World Knowledge for Open-World Embodied Reasoningcited by 2
  13. 2026How Far Are Vision-Language Models from Constructing the Real World? A Benchmark for Physical Generative Reasoningcited by 2
  14. 2026EgoProx: Evaluating MLLMs on Egocentric 3D Proximity Reasoning Across a Cognitive Hierarchycited by 2
  15. 2025Context-Aware Sentiment Forecasting via LLM-based Multi-Perspective Role-Playing Agentscited by 1
  16. 2026From Pixels to Policies: Reinforcing Spatial Reasoning in Language Models for Content-Aware Layout Designcited by 1
  17. 2026Perceive-then-Plan: Layout-as-Policy for Monocular 3D Scene Layout Estimationcited by 1
  18. 2026PAR3D: A Unified 3D-MLLM with Part-Aware Representation for Scene Understandingcited by 1
  19. 2026OmniView-Space: Reinforcing Spatial Reasoning via Multi-Perspective Spatial Mappingcited by 1
  20. 2025A Study of the Framework and Real-World Applications of Language Embedding for 3D Scene Understandingcited by 0
  21. 2025Pay Less Attention to Function Words for Free Robustness of Vision-Language Modelscited by 0
  22. 2025The N-Body Problem: Parallel Execution from Single-Person Egocentric Videocited by 0
  23. 2026Ground-Truth Depth in Vision Language Models: Spatial Context Understanding in Conversational AI for XR-Robotic Support in Emergency First Responsecited by 0
  24. 2026Learning Multi-View Spatial Reasoning from Cross-View Relationscited by 0
  25. 2026How Far Are Large Multimodal Models from Human-Level Spatial Action? A Benchmark for Goal-Oriented Embodied Navigation in Urban Airspacecited by 0
  26. 2026Holo-Captioning: Toward the Text Equivalent of 3D Scenescited by 0
  27. 2026IDEAL-Bench: Indoor Dataset and Evaluation suite for Analyzing 3D Layout reasoningcited by 0
  28. 2026Quo Vadis, World Modeling?cited by 0
  29. 2026SmartMage: Dynamic Modality Orchestration for 3D Scene Understandingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Spatial 3D-LLM: Exploring Spatial Awareness in 3D Vision-Language Models
  2. 20233D-LLM: Injecting the 3D World into Large Language Models
  3. 20253UR-LLM: An End-to-End Multimodal Large Language Model for 3D Scene Understanding
  4. 2025The Point, the Vision and the Text: Does Point Cloud Boost Spatial Reasoning of Large Language Models? A Bias-Controlled Study
  5. 2026Can LLMs See Without Pixels? Benchmarking Spatial Intelligence from Textual Descriptions
  6. 2024SPARTUN3D: Situated Spatial Understanding of 3D World in Large Language Models
  7. 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding
  8. 2026Do 3D Large Language Models Really Understand 3D Spatial Relationships?