The Bare Necessities: Designing Simple, Effective Open-Vocabulary Scene Graphs
Not on the map. Why?
Abstract (arXiv)
3D open-vocabulary scene graph methods are a promising map representation for embodied agents, however many current approaches are computationally expensive. In this paper, we reexamine the critical design choices established in previous works to optimize both efficiency and performance. We propose a general scene graph framework and conduct three studies that focus on image pre-processing, feature fusion, and feature selection. Our findings reveal that commonly used image pre-processing techniques provide minimal performance improvement while tripling computation (on a per object view basis). We also show that averaging feature labels across different views significantly degrades performance. We study alternative feature selection strategies that enhance performance without adding unnecessary computational costs. Based on our findings, we introduce a computationally balanced approach for 3D point cloud segmentation with per-object features. The approach matches state-of-the-art classification accuracy while achieving a threefold reduction in computation.
Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 29 of its 44 references that are in the map. First posted on arXiv 2 Dec 2024.
Builds on
29 of 44 references.
References · 29
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023Visual Instruction Tuningcited by 7,792
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
- 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
- 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
- 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
- 2023Faster Segment Anything: Towards Lightweight SAM for Mobile Applicationscited by 345
- 2020PointGroup: Dual-Set Point Grouping for 3D Instance Segmentationcited by 318
- 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentationcited by 299
- 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 288
- 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 231
- 2023What does CLIP know about a red circle? Visual prompt engineering for VLMscited by 215
17 more
- 2022SoftGroup for 3D Instance Segmentation on Point Cloudscited by 197
- 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 195
- 2019Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mappingcited by 181
- 2020OccuSeg: Occupancy-aware 3D Instance Segmentationcited by 177
- 20203D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentationcited by 140
- 2023SAI3D: Segment Any Instance in 3D Scenescited by 90
- 2024Clio: Real-time Task-Driven Open-Set 3D Scene Graphscited by 81
- 2024Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationshipscited by 76
- 2023Foundations of Spatial Perception for Robotics: Hierarchical Representations and Real-time Systemscited by 60
- 2024RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulationcited by 48
- 2018FutureMapping: The Computational Structure of Spatial AI Systemscited by 35
- 2019Bayesian Spatial Kernel Smoothing for Scalable Dense Semantic Mappingcited by 34
- 2023Language-EXtended Indoor SLAM (LEXIS): A Versatile System for Real-time Visual Scene Understandingcited by 15
- 2023LABELMAKER: Automatic Semantic Label Generation from RGB-D Trajectoriescited by 14
- 2023MV-CLIP: Multi-View CLIP for Zero-shot 3D Shape Recognitioncited by 11
- 2023Active Open-Vocabulary Recognition: Let Intelligent Moving Mitigate CLIP Limitationscited by 6
- 2024Deep Learning-Based 3D Instance and Semantic Segmentation: A Reviewcited by 1
Built on by
Citing papers per year: 2025: 7; 2026: 4.
11 papers.
- 2025FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interactioncited by 22
- 2025Describe Anything Anywhere At Any Momentcited by 19
- 2025GraphPilot: Grounded Scene Graph Conditioning for Language-Based Autonomous Drivingcited by 6
- 2025Bayesian Fields: Task-driven Open-Set Semantic Gaussian Splattingcited by 5
- 2025Object-Centric Representation Learning for Enhanced 3D Semantic Scene Graph Predictioncited by 2
- 2025QueryAdapter: Rapid Adaptation of Vision-Language Models in Response to Natural Language Queriescited by 1
- 2025OpenLex3D: A Tiered Evaluation Benchmark for Open-Vocabulary 3D Scene Representationscited by 1
- 2026LIEREx: Language-Image Embeddings for Robotic Explorationcited by 1
- 2026DISC: Dense Integrated Semantic Context for Large-Scale Open-Set Semantic Mappingcited by 1
- 2026Prior-SG: Task and Prior Driven Region Segmentation for Scene Graphs in Arbitrarily-Structured Environmentscited by 0
- 2026TRACKGRAPH: Online Open-Vocabulary 3D Scene Graphs via Image-Space Trackingcited by 0
- 2026TRACKGRAPH: Online Open-Vocabulary 3D Scene Graphs via Image-Space Tracking25 Sep 2026
- 2026Prior-SG: Task and Prior Driven Region Segmentation for Scene Graphs in Arbitrarily-Structured Environments6 Aug 2026
- 2026DISC: Dense Integrated Semantic Context for Large-Scale Open-Set Semantic Mapping4 Mar 2026
- 2026LIEREx: Language-Image Embeddings for Robotic Exploration2 Feb 2026
- 2025Describe Anything Anywhere At Any Moment29 Nov 2025
- 2025GraphPilot: Grounded Scene Graph Conditioning for Language-Based Autonomous Driving14 Nov 2025
- 2025Object-Centric Representation Learning for Enhanced 3D Semantic Scene Graph Prediction6 Oct 2025
- 2025OpenLex3D: A Tiered Evaluation Benchmark for Open-Vocabulary 3D Scene Representations25 Mar 2025
- 2025FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interaction10 Mar 2025
- 2025Bayesian Fields: Task-driven Open-Set Semantic Gaussian Splatting7 Mar 2025
- 2025QueryAdapter: Rapid Adaptation of Vision-Language Models in Response to Natural Language Queries26 Feb 2025