mlsanityData to 14 Oct 2026

3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera

arXiv 1910.02527 ↗First posted 6 Oct 2019cs.CV cs.ROcited by 281

Not on the map. Why?

Abstract (arXiv)

A comprehensive semantic understanding of a scene is important for many applications - but in what space should diverse semantic information (e.g., objects, scene categories, material types, texture, etc.) be grounded and what should be its structure? Aspiring to have one unified structure that hosts diverse types of semantics, we follow the Scene Graph paradigm in 3D, generating a 3D Scene Graph. Given a 3D mesh and registered panoramic images, we construct a graph that spans the entire building and includes semantics on objects (e.g., class, material, and other attributes), rooms (e.g., scene category, volume, etc.) and cameras (e.g., location, etc.), as well as the relationships among these entities. However, this process is prohibitively labor heavy if done manually. To alleviate this we devise a semi-automatic framework that employs existing detection methods and enhances them using two main constraints: I. framing of query images sampled on panoramas to maximize the performance of 2D detectors, and II. multi-view consistency enforcement across 2D detections that originate in different camera locations.

Key facts. Cited by 281 papers in the map (data through 14 Oct 2026). First posted on arXiv 6 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 27; 2022: 41; 2023: 35; 2024: 46; 2025: 56; 2026: 76.

281 papers.

Sort:

By area:

  1. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 365
  2. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  3. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 294
  4. 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 196
  5. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 178
  6. 2021SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequencescited by 163
  7. 2022Open-vocabulary Queryable Scene Representations for Real World Planningcited by 160
  8. 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 150
  9. 2021Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphscited by 144
  10. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 133
  11. 2021A Comprehensive Survey of Scene Graphs: Generation and Applicationcited by 125
  12. 2021SAT: 2D Semantics Assisted Training for 3D Visual Groundingcited by 124
38 more
  1. 2021NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenescited by 119
  2. 20223D Common Corruptions and Data Augmentationcited by 119
  3. 2022Habitat-Matterport 3D Semantics Datasetcited by 110
  4. 2024I-Design: Personalized LLM Interior Designercited by 87
  5. 2021Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphscited by 84
  6. 2024Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationshipscited by 80
  7. 20243D-Mem: 3D Scene Memory for Embodied Exploration and Reasoningcited by 75
  8. 2022Scene Graph Generation: A Comprehensive Surveycited by 73
  9. 2021Recovering the Unbiased Scene Graphs from the Biased Onescited by 71
  10. 2022Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigationcited by 67
  11. 2022Relationformer: A Unified Framework for Image-to-Graph Generationcited by 67
  12. 2021Learning to Generate Scene Graph from Natural Language Supervisioncited by 64
  13. 2021Learning 3D Semantic Segmentation with only 2D Image Supervisioncited by 60
  14. 2022TASKOGRAPHY: Evaluating robot task planning over large 3D scene graphscited by 59
  15. 2022Housekeep: Tidying Virtual Households using Commonsense Reasoningcited by 57
  16. 2023NS3D: Neuro-Symbolic Grounding of 3D Objects and Relationscited by 57
  17. 2025UniGoal: Towards Universal Zero-shot Goal-oriented Navigationcited by 57
  18. 2022Towards Open-vocabulary Scene Graph Generation with Prompt-based Finetuningcited by 54
  19. 2023CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusioncited by 54
  20. 2023How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontierscited by 54
  21. 2025Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spacescited by 54
  22. 2023SayNav: Grounding Large Language Models for Dynamic Planning to Navigation in New Environmentscited by 53
  23. 2021Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networkscited by 52
  24. 20224D-OR: Semantic Scene Graphs for OR Domain Modelingcited by 49
  25. 2021SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistencycited by 48
  26. 2024RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulationcited by 48
  27. 2022Learning Multi-View Aggregation In the Wild for Large-Scale 3D Semantic Segmentationcited by 47
  28. 2022MORE: Multi-Order RElation Mining for Dense Captioning in 3D Scenescited by 46
  29. 2023Nerflets: Local Radiance Fields for Efficient Structure-Aware 3D Scene Representation from 2D Supervisioncited by 44
  30. 2023VL-SAT: Visual-Linguistic Semantics Assisted Training for 3D Semantic Scene Graph Prediction in Point Cloudcited by 43
  31. 2022Continuous Scene Representations for Embodied AIcited by 39
  32. 2023SceneHGN: Hierarchical Graph Networks for 3D Indoor Scene Generation with Fine-Grained Geometrycited by 39
  33. 2023Incremental 3D Semantic Scene Graph Prediction from RGB Sequencescited by 39
  34. 2024KARMA: Augmenting Embodied AI Agents with Long-and-short Term Memory Systemscited by 39
  35. 2022LaTeRF: Label and Text Driven Object Radiance Fieldscited by 38
  36. 2024GraphEQA: Using 3D Semantic Scene Graphs for Real-time Embodied Question Answeringcited by 37
  37. 2021From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligencecited by 35
  38. 2024Dynamic 3D Gaussian Fields for Urban Areascited by 35

50 most cited of 281 All 281 (JSON)