SGR3 Model: Scene Graph Retrieval-Reasoning Model in 3D
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
3D scene graphs provide a structured representation of object entities and their relationships, enabling high-level interpretation and reasoning for robots while remaining intuitively understandable to humans. Existing approaches for 3D scene graph generation typically combine scene reconstruction with graph neural networks (GNNs). However, such pipelines require multi-modal data that may not always be available, and their reliance on heuristic graph construction can constrain the prediction of relationship triplets. In this work, we introduce a Scene Graph Retrieval-Reasoning Model in 3D (SGR3 Model), a training-free framework that leverages multi-modal large language models (MLLMs) with retrieval-augmented generation (RAG) for semantic scene graph generation. SGR3 Model bypasses the need for explicit 3D reconstruction. Instead, it enhances relational reasoning by incorporating semantically aligned scene graphs retrieved via a ColPali-style cross-modal framework. To improve retrieval robustness, we further introduce a weighted patch-level similarity selection mechanism that mitigates the negative impact of blurry or semantically uninformative regions. Experiments demonstrate that SGR3 Model achieves competitive performance compared to training-free baselines and on par with GNN-based expert models. Moreover, an ablation study on the retrieval module and knowledge base scale reveals that retrieved external information is explicitly integrated into the token generation process, rather than being implicitly internalized through abstraction.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 7 of its 42 references that are in the map. First posted on arXiv 4 Mar 2026.
Builds on
7 of 42 references.
Closest ancestors · 1
Same direction, newest first
Other references · 6
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
- 2025SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognitioncited by 35
- 2025Open World Scene Graph Generation using Vision Language Modelscited by 7
- 2026INHerit-SG: Incremental Hierarchical Semantic Scene Graphs for Complex Embodied Retrievalcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 20243DGraphLLM: Combining Semantic Graphs and Large Language Models for 3D Scene Understanding
- 2026RAG-3DSG: Enhancing 3D Scene Graphs with Re-Shot Guided Retrieval-Augmented Generation
- 2026CAPruner: Conceptual-Adjacent Scene Graph Pruner for Enhancing 3D Spatial Reasoning of Large Language Models
- 2024EmbodiedRAG: Dynamic 3D Scene Graph Retrieval for Efficient and Scalable Robot Task Planning
- 2026DeWorldSG: Depth-Aware 3D Semantic Scene Graph Generation via World-Model Priors
- 2026Relationship-Aware Hierarchical 3D Scene Graph for Task Reasoning
- 2026From Pixels to Concepts: Growing Rich 3D Semantic Scene Graph Forests utilizing Foundation Models ↗
- 2025Sparse3DPR: Training-Free 3D Hierarchical Scene Parsing and Task-Adaptive Subgraph Reasoning from Sparse RGB Views