mlsanityData to 7 Oct 2026

SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences

arXiv 2103.14898 ↗First posted 27 Mar 2021cs.CV cs.LGcited by 159

Not on the map. Why?

Abstract (arXiv)

Scene graphs are a compact and explicit representation successfully used in a variety of 2D scene understanding tasks. This work proposes a method to incrementally build up semantic scene graphs from a 3D environment given a sequence of RGB-D frames. To this end, we aggregate PointNet features from primitive scene components by means of a graph neural network. We also propose a novel attention mechanism well suited for partial and missing graph data present in such an incremental reconstruction scenario. Although our proposed method is designed to run on submaps of the scene, we show it also transfers to entire 3D scenes. Experiments show that our approach outperforms 3D scene graph prediction methods by a large margin and its accuracy is on par with other 3D semantic and panoptic segmentation methods while running at 35 Hz.

Key facts. Cited by 159 papers in the map (data through 7 Oct 2026). Builds on 49 of its 64 references that are in the map. First posted on arXiv 27 Mar 2021.

Builds on

49 of 64 references.

References · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Graph Attention Networkscited by 8,854
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  7. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  8. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  9. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  10. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  11. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 966
  12. 2019On the Convergence of Adam and Beyondcited by 834
37 more
  1. 2018Panoptic Segmentationcited by 796
  2. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  3. 2017Scene Graph Generation by Iterative Message Passingcited by 462
  4. 2018Image Generation from Scene Graphscited by 324
  5. 2018Graph R-CNN for Scene Graph Generationcited by 296
  6. 20193D Scene Graph: A Structure for Unified Semantics, 3D Space, and Cameracited by 280
  7. 20183D-SIS: 3D Semantic Instance Segmentation of RGB-D Scanscited by 216
  8. 2020Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructionscited by 205
  9. 2019RIO: 3D Object Instance Re-Localization in Changing Indoor Environmentscited by 187
  10. 2019Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mappingcited by 181
  11. 2020OccuSeg: Occupancy-aware 3D Instance Segmentationcited by 177
  12. 2017Scene Graph Generation from Objects, Phrases and Region Captionscited by 154
  13. 2017CNN-SLAM: Real-time dense monocular SLAM with learned depth predictioncited by 152
  14. 20203D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humanscited by 141
  15. 20203D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentationcited by 140
  16. 2016SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networkscited by 134
  17. 2016Graph-Structured Representations for Visual Question Answeringcited by 123
  18. 2019Scene Graph Generation with External Knowledge and Image Reconstructioncited by 102
  19. 2017Detecting Visual Relationships with Deep Relational Networkscited by 97
  20. 2018Fusion++: Volumetric Object-Level SLAMcited by 97
  21. 2019PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Thingscited by 97
  22. 2017Dynamic Graph Convolutional Networkscited by 83
  23. 2020Semantic Image Manipulation Using Scene Graphscited by 82
  24. 2019Volumetric Instance-Aware Semantic Mapping and 3D Object Discoverycited by 79
  25. 20193D Instance Segmentation via Multi-Task Metric Learningcited by 79
  26. 2018Attentive Relational Networks for Mapping Images to Scene Graphscited by 72
  27. 2017Pixels to Graphs by Associative Embeddingcited by 58
  28. 2018TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshescited by 45
  29. 2018Fully-Convolutional Point Networks for Large-Scale Point Cloudscited by 44
  30. 2020Fusion-Aware Point Convolution for Online Semantic 3D Scene Segmentationcited by 40
  31. 2020Latent-Graph Learning for Disease Predictioncited by 39
  32. 2018Exploring Spatial Context for 3D Semantic Segmentation of Point Cloudscited by 33
  33. 2018Visual Graphs from Motion (VGfM): Scene understanding with object geometry reasoningcited by 27
  34. 2018Real-time Progressive 3D Semantic Segmentation for Indoor Scenecited by 18
  35. 2018Incremental Object Database: Building 3D Models from Multiple Partial Observationscited by 15
  36. 2015SemanticPaint: A Framework for the Interactive Segmentation of 3D Scenescited by 8
  37. 2018Fast and Accurate Semantic Mapping through Geometric-based Incremental Segmentationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 19; 2023: 26; 2024: 30; 2025: 30; 2026: 47.

159 papers.

Sort:

By area:

  1. 2022Visual Language Maps for Robot Navigationcited by 362
  2. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 357
  3. 2023An Embodied Generalist Agent in 3D Worldcited by 355
  4. 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 201
  5. 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 195
  6. 2023LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmarkcited by 169
  7. 2022Open-vocabulary Queryable Scene Representations for Real World Planningcited by 159
  8. 2023OneFormer3D: One Transformer for Unified Point Cloud Segmentationcited by 95
  9. 2021Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphscited by 83
  10. 2024Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationshipscited by 76
  11. 20243D-Mem: 3D Scene Memory for Embodied Exploration and Reasoningcited by 73
  12. 2023CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusioncited by 54
38 more
  1. 2023SayNav: Grounding Large Language Models for Dynamic Planning to Navigation in New Environmentscited by 53
  2. 2025Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spacescited by 53
  3. 2025Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigationcited by 50
  4. 2022MORE: Multi-Order RElation Mining for Dense Captioning in 3D Scenescited by 46
  5. 2023VL-SAT: Visual-Linguistic Semantics Assisted Training for 3D Semantic Scene Graph Prediction in Point Cloudcited by 42
  6. 2023Incremental 3D Semantic Scene Graph Prediction from RGB Sequencescited by 39
  7. 2022Latent Graph Representations for Critical View of Safety Assessmentcited by 37
  8. 2024GraphEQA: Using 3D Semantic Scene Graphs for Real-time Embodied Question Answeringcited by 37
  9. 2024DELTA: Decomposed Efficient Long-Term Robot Task Planning using Large Language Modelscited by 33
  10. 2023SGAligner : 3D Scene Alignment with Scene Graphscited by 30
  11. 2022(2.5+1)D Spatio-Temporal Scene Graphs for Video Question Answeringcited by 29
  12. 2024SceneGraphLoc: Cross-Modal Coarse Visual Localization on 3D Scene Graphscited by 29
  13. 2025What Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation Modelscited by 29
  14. 2022S-Graphs+: Real-time Localization and Mapping leveraging Hierarchical Representationscited by 28
  15. 2024SPARTUN3D: Situated Spatial Understanding of 3D World in Large Language Modelscited by 26
  16. 2023Indoor and Outdoor 3D Scene Graph Generation via Language-Enabled Spatial Ontologiescited by 25
  17. 2022Bending Graphs: Hierarchical Shape Matching using Gated Optimal Transportcited by 24
  18. 20223D VSG: Long-term Semantic Scene Change Prediction through 3D Variable Scene Graphscited by 24
  19. 2023SG-Bot: Object Rearrangement via Coarse-to-Fine Robotic Imagination on Scene Graphscited by 24
  20. 2021Unconditional Scene Graph Generationcited by 22
  21. 2024OpenGraph: Open-Vocabulary Hierarchical 3D Graph Representation in Large-Scale Outdoor Environmentscited by 22
  22. 20244D Panoptic Scene Graph Generationcited by 22
  23. 2025FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interactioncited by 22
  24. 2022Situational Graphs for Robot Navigation in Structured Indoor Environmentscited by 21
  25. 2023A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Applicationcited by 21
  26. 2023SGRec3D: Self-Supervised 3D Scene Graph Learning via Object-Level Scene Reconstructioncited by 20
  27. 2025Describe Anything Anywhere At Any Momentcited by 19
  28. 2023Lang3DSG: Language-based contrastive pre-training for 3D Scene Graph predictioncited by 18
  29. 2021Comprehensive Visual Question Answering on Point Clouds through Compositional Scene Manipulationcited by 17
  30. 2022Sequential Manipulation Planning on Scene Graphcited by 16
  31. 2024RelationField: Relate Anything in Radiance Fieldscited by 16
  32. 2023An Object SLAM Framework for Association, Mapping, and High-Level Taskscited by 15
  33. 2024A Unified Framework for 3D Scene Understandingcited by 15
  34. 2021From SLAM to Situational Awareness: Challenges and Surveycited by 14
  35. 2024Scalable 3D Panoptic Segmentation As Superpoint Graph Clusteringcited by 14
  36. 2026Multiplayer Interactive World Models with Representation Autoencoderscited by 14
  37. 2025Universal Scene Graph Generationcited by 11
  38. 2023Volumetric Semantically Consistent 3D Panoptic Mappingcited by 10

50 most cited of 159 All 159 (JSON)