mlsanityData to 7 Oct 2026

SSR3D-LLM: Structured Spatial Reasoning via Latent Steps for Fine-Grained Grounding in Unified 3D-LLMs

arXiv 2605.28490 ↗First posted 27 May 2026cs.CV cs.AIcited by 0

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

3D object grounding localizes referred objects in a 3D scene from natural language. Unified instance-centric 3D-LLMs aim to solve grounding together with dialog, QA, and captioning, yet many rely on a single pointer-style grounding decision that compresses a relational instruction into one selection. This is brittle for fine-grained queries where multiple same-class candidates must be ruled out by context objects and spatial relations. We propose Structured Spatial Reasoning 3D-LLM (SSR3D-LLM), a structured grounding interface for unified 3D-LLMs. Given fixed Mask3D object proposals, the LLM writes a sequence of latent spatial reasoning steps and memory tokens from the query, and a geometry-aware scorer reads these latent steps in order to refine candidate rankings step by step with step-length masking. The latent steps are learned from standard benchmark target supervision with auxiliary referential-cue supervision during training, while inference uses only the input query and Mask3D proposals. Across ReferIt3D, ScanRefer, and Multi3DRef, SSR3D-LLM achieves the strongest results among unified 3D-LLM baselines, with substantial gains over the single-pointer QPG baseline on fine-grained grounding and consistent improvements over prior unified 3D-LLMs, while preserving the default language-task route.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 34 of its 40 references that are in the map. First posted on arXiv 27 May 2026.

Builds on

34 of 40 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods14 Nov
  2. 2025SPAZER: Spatial-Semantic Progressive Reasoning Agent for Zero-shot 3D Visual Grounding27 Jun
  3. 2025SORT3D: Spatial Object-centric Reasoning Toolbox for Zero-Shot 3D Grounding Using Large Language Models25 Apr
  4. 2023CoT3DRef: Chain-of-Thoughts Data-Efficient 3D Visual Grounding10 Oct

Other references · 30

  1. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  2. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  3. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
  4. 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentationcited by 299
  5. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
  6. 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 170
  7. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 142
  8. 2022Multi-View Transformer for 3D Visual Groundingcited by 138
  9. 2023LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agentcited by 134
  10. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 130
  11. 2022Language Conditioned Spatial Relation Reasoning for 3D Object Groundingcited by 121
  12. 2024Grounded 3D-LLM with Referent Tokenscited by 111
18 more
  1. 20223D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selectioncited by 109
  2. 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Modelscited by 106
  3. 2021LanguageRefer: Spatial-Language Model for 3D Visual Groundingcited by 88
  4. 2021TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Groundingcited by 86
  5. 2023ViewRefer: Grasp the Multi-view Knowledge for 3D Visual Grounding with GPT and Prototype Guidancecited by 70
  6. 2024SeeGround: See and Ground for Zero-Shot Open-Vocabulary 3D Visual Groundingcited by 66
  7. 2023Visual Programming for Zero-shot Open-Vocabulary 3D Visual Groundingcited by 64
  8. 2024VLM-Grounder: A VLM Agent for Zero-Shot 3D Visual Groundingcited by 56
  9. 20243D-GRAND: A Million-Scale Dataset for 3D-LLMs with Better Grounding and Less Hallucinationcited by 36
  10. 2022ScanEnts3D: Exploiting Phrase-to-3D-Object Correspondences for Improved Visio-Linguistic Models in 3D Scenescited by 32
  11. 2024MiKASA: Multi-Key-Anchor & Scene-Aware Transformer for 3D Visual Groundingcited by 29
  12. 2024A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future Directionscited by 24
  13. 2024Dense Multimodal Alignment for Open-Vocabulary 3D Scene Understandingcited by 15
  14. 2024A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenescited by 10
  15. 2024Transcrib3D: 3D Referring Expression Resolution through Large Language Modelscited by 10
  16. 2025Language-to-Space Programming for Training-Free 3D Visual Groundingcited by 7
  17. 2025Better, Stronger, Faster: Tackling the Trilemma in MLLM-based Segmentation with Simultaneous Textual Mask Predictioncited by 7
  18. 2024Data-Efficient 3D Visual Grounding via Order-Aware Referringcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Neural Representation Framework with LLM-Driven Spatial Reasoning for Open-Vocabulary 3D Visual Grounding
  2. 2025Reasoning in Space via Grounding in the World
  3. 2026Ground3D-LMM: Fine-Grained 3D Point Grounding and Spatial Reasoning with LMM
  4. 2026Grounded 3D-Aware Spatial Vision-Language Modeling
  5. 2025S2-MLLM: Boosting Spatial Reasoning Capability of MLLMs for 3D Visual Grounding with Structural Guidance
  6. 2025From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes
  7. 2025N3D-VLM: Native 3D Grounding Enables Accurate Spatial Reasoning in Vision-Language Models
  8. 2026UniGround: Universal 3D Visual Grounding via Training-Free Scene Parsing