mlsanityData to 7 Oct 2026

Beyond Isolated Objects: Relationship-aware Open Vocabulary Scene Understanding via 3D Scene Graph Analysis

arXiv 2607.05348 ↗First posted 6 Jul 2026cs.CV cs.ROcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Open-vocabulary 3D scene understanding aims to segment 3D scenes beyond predefined categories by transferring semantic knowledge from vision-language models. Existing methods have advanced this task by lifting language-aligned 2D features into 3D, yet they often rely on context-independent semantic representations, leaving object relationships underexplored for contextual refinement. We propose RelGraphOV, a relationship-aware framework that uses 3D scene graphs to enhance open-vocabulary 3D understanding. Our method constructs relational scene graphs from multi-view observations by leveraging vision-language reasoning to infer object relationships and prune geometrically implausible connections, without manual relationship annotations. To aggregate relational context while avoiding feature interference, we introduce an Adaptive Gated Dual-Stream Contextual GAT that separates dense geometric features and semantic CLIP embeddings, performs edge-guided message passing, and adaptively fuses complementary semantics. A hierarchical contrastive objective further promotes instance-level consistency and category-level discrimination. Experiments on ScanNetV2, ScanNet200, ScanNet++, and Replica demonstrate strong performance and generalization ability. Project Page: https://cxavireh.github.io/relgraphov-projectpage

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 45 references that are in the map. First posted on arXiv 6 Jul 2026.

Builds on

35 of 45 references.

References · 35

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  3. 2022Scalable Diffusion Models with Transformerscited by 5,931
  4. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  5. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  6. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  7. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  8. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  9. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  10. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  11. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  12. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
23 more
  1. 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 667
  2. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 553
  3. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
  4. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  5. 2023FSGS: Real-Time Few-shot View Synthesis using Gaussian Splattingcited by 278
  6. 2024No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Imagescited by 241
  7. 2025AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Viewscited by 234
  8. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 222
  9. 2024WildGaussians: 3D Gaussian Splatting in the Wildcited by 150
  10. 2024Gaussian in the Wild: 3D Gaussian Splatting for Unconstrained Image Collectionscited by 117
  11. 2023Learning to Render Novel Views from Wide-Baseline Stereo Pairscited by 103
  12. 2023GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformerscited by 75
  13. 2023MuRF: Multi-Baseline Radiance Fieldscited by 70
  14. 2024SWAG: Splatting in the Wild images with Appearance-conditioned Gaussianscited by 61
  15. 2025WorldMirror: Universal 3D World Reconstruction with Any-Prior Promptingcited by 60
  16. 2024MegaScenes: Scene-Level View Synthesis at Scalecited by 58
  17. 2024Wild-GS: Real-Time Novel View Synthesis from Unconstrained Photo Collectionscited by 56
  18. 2023Explicit Correspondence Matching for Generalizable Neural Radiance Fieldscited by 46
  19. 2024SplatLoc: 3D Gaussian Splatting-based Visual Localization for Augmented Realitycited by 42
  20. 2024Splatfacto-W: A Nerfstudio Implementation of Gaussian Splatting for Unconstrained Photo Collectionscited by 28
  21. 2025TC-Light: Temporally Coherent Generative Rendering for Realistic World Transfercited by 11
  22. 2025AtlasGS: Atlanta-world Guided Surface Reconstruction with Implicit Structured Gaussianscited by 4
  23. 2026OnlinePG: Online Open-Vocabulary Panoptic Mapping with 3D Gaussian Splattingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 20243DGraphLLM: Combining Semantic Graphs and Large Language Models for 3D Scene Understanding
  2. 2024Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationships
  3. 2026Relationship-Aware Hierarchical 3D Scene Graph for Task Reasoning
  4. 2024Open-Vocabulary Octree-Graph for 3D Scene Understanding
  5. 2023Lang3DSG: Language-based contrastive pre-training for 3D Scene Graph prediction
  6. 2025FreeQ-Graph: Free-form Querying with Semantic Consistent Scene Graph for 3D Scene Understanding
  7. 2026Unveiling the Unknown: Open Vocabulary Object Detection with Scene Graphs
  8. 2026SceneScaffold: Active Scene-State Construction for Unified 3D Scene Understanding