mlsanityData to 7 Oct 2026

The Bare Necessities: Designing Simple, Effective Open-Vocabulary Scene Graphs

arXiv 2412.01539 ↗First posted 2 Dec 2024cs.CV cs.ROcited by 11

Not on the map. Why?

Abstract (arXiv)

3D open-vocabulary scene graph methods are a promising map representation for embodied agents, however many current approaches are computationally expensive. In this paper, we reexamine the critical design choices established in previous works to optimize both efficiency and performance. We propose a general scene graph framework and conduct three studies that focus on image pre-processing, feature fusion, and feature selection. Our findings reveal that commonly used image pre-processing techniques provide minimal performance improvement while tripling computation (on a per object view basis). We also show that averaging feature labels across different views significantly degrades performance. We study alternative feature selection strategies that enhance performance without adding unnecessary computational costs. Based on our findings, we introduce a computationally balanced approach for 3D point cloud segmentation with per-object features. The approach matches state-of-the-art classification accuracy while achieving a threefold reduction in computation.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 29 of its 44 references that are in the map. First posted on arXiv 2 Dec 2024.

Builds on

29 of 44 references.

References · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  4. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  5. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  6. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
  7. 2023Faster Segment Anything: Towards Lightweight SAM for Mobile Applicationscited by 345
  8. 2020PointGroup: Dual-Set Point Grouping for 3D Instance Segmentationcited by 318
  9. 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentationcited by 299
  10. 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 288
  11. 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 231
  12. 2023What does CLIP know about a red circle? Visual prompt engineering for VLMscited by 215
17 more
  1. 2022SoftGroup for 3D Instance Segmentation on Point Cloudscited by 197
  2. 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 195
  3. 2019Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mappingcited by 181
  4. 2020OccuSeg: Occupancy-aware 3D Instance Segmentationcited by 177
  5. 20203D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentationcited by 140
  6. 2023SAI3D: Segment Any Instance in 3D Scenescited by 90
  7. 2024Clio: Real-time Task-Driven Open-Set 3D Scene Graphscited by 81
  8. 2024Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationshipscited by 76
  9. 2023Foundations of Spatial Perception for Robotics: Hierarchical Representations and Real-time Systemscited by 60
  10. 2024RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulationcited by 48
  11. 2018FutureMapping: The Computational Structure of Spatial AI Systemscited by 35
  12. 2019Bayesian Spatial Kernel Smoothing for Scalable Dense Semantic Mappingcited by 34
  13. 2023Language-EXtended Indoor SLAM (LEXIS): A Versatile System for Real-time Visual Scene Understandingcited by 15
  14. 2023LABELMAKER: Automatic Semantic Label Generation from RGB-D Trajectoriescited by 14
  15. 2023MV-CLIP: Multi-View CLIP for Zero-shot 3D Shape Recognitioncited by 11
  16. 2023Active Open-Vocabulary Recognition: Let Intelligent Moving Mitigate CLIP Limitationscited by 6
  17. 2024Deep Learning-Based 3D Instance and Semantic Segmentation: A Reviewcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 4.

11 papers.