mlsanityData to 7 Oct 2026

Scene Graph Generation: A Comprehensive Survey

arXiv 2201.00443 ↗First posted 3 Jan 2022cs.CVcited by 73

Not on the map. Why?

Abstract (arXiv)

Deep learning techniques have led to remarkable breakthroughs in the field of generic object detection and have spawned a lot of scene-understanding tasks in recent years. Scene graph has been the focus of research because of its powerful semantic representation and applications to scene understanding. Scene Graph Generation (SGG) refers to the task of automatically mapping an image into a semantic structural scene graph, which requires the correct labeling of detected objects and their relationships. Although this is a challenging task, the community has proposed a lot of SGG approaches and achieved good results. In this paper, we provide a comprehensive survey of recent achievements in this field brought about by deep learning techniques. We review 138 representative works that cover different input modalities, and systematically summarize existing methods of image-based SGG from the perspective of feature extraction and fusion. We attempt to connect and systematize the existing visual relationship detection methods, to summarize, and interpret the mechanisms and the strategies of SGG in a comprehensive way. Finally, we finish this survey with deep discussions about current existing problems and future research directions. This survey will help readers to develop a better understanding of the current research status and ideas.

Key facts. Cited by 73 papers in the map (data through 7 Oct 2026). Builds on 133 of its 208 references that are in the map. First posted on arXiv 3 Jan 2022.

Builds on

133 of 208 references.

References · 133

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2017Graph Attention Networkscited by 8,854
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  8. 2016Densely Connected Convolutional Networkscited by 6,691
  9. 2017Inductive Representation Learning on Large Graphscited by 6,284
  10. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  11. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  12. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
38 more
  1. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  2. 2018YOLOv3: An Incremental Improvementcited by 2,652
  3. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  4. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
  5. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  6. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  7. 2019Objects as Pointscited by 1,172
  8. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 939
  9. 2018Panoptic Segmentationcited by 796
  10. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
  11. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 542
  12. 2019Hybrid Task Cascade for Instance Segmentationcited by 479
  13. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  14. 2017Scene Graph Generation by Iterative Message Passingcited by 462
  15. 2017Neural Motifs: Scene Graph Parsing with Global Contextcited by 413
  16. 2019From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Networkcited by 366
  17. 2019PIXOR: Real-time 3D Object Detection from Point Cloudscited by 358
  18. 2020Unbiased Scene Graph Generation from Biased Trainingcited by 355
  19. 2014Recurrent Models of Visual Attentioncited by 344
  20. 2016Visual Relationship Detection with Language Priorscited by 333
  21. 2018Image Generation from Scene Graphscited by 324
  22. 2015Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networkscited by 305
  23. 2017Relation Networks for Object Detectioncited by 305
  24. 2018Graph R-CNN for Scene Graph Generationcited by 296
  25. 20193D Scene Graph: A Structure for Unified Semantics, 3D Space, and Cameracited by 280
  26. 2018Learning to Compose Dynamic Tree Structures for Visual Contextscited by 258
  27. 2018Auto-Encoding Scene Graphs for Image Captioningcited by 254
  28. 2019Action Genome: Actions as Composition of Spatio-temporal Scene Graphscited by 249
  29. 2015Conditional Random Fields as Recurrent Neural Networkscited by 213
  30. 2020Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructionscited by 205
  31. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  32. 2018Learning Human-Object Interactions by Graph Parsing Neural Networkscited by 176
  33. 2020GPS-Net: Graph Property Sensing Network for Scene Graph Generationcited by 165
  34. 2019Knowledge-Embedded Routing Network for Scene Graph Generationcited by 163
  35. 2017Scene Graph Generation from Objects, Phrases and Region Captionscited by 154
  36. 20203D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humanscited by 141
  37. 2019xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentationcited by 129
  38. 2018Grounded Human-Object Interaction Hotspots from Videocited by 128

50 most cited of 133 All 133 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 15; 2024: 22; 2025: 17; 2026: 15.

73 papers.

Sort:

By area:

  1. 2022A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspectivecited by 56
  2. 2024Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Modelscited by 30
  3. 2023HiLo: Exploiting High Low Frequency Relations for Unbiased Panoptic Scene Graph Generationcited by 19
  4. 2024Emergent Visual-Semantic Hierarchies in Image-Text Representationscited by 19
  5. 2023Pair then Relation: Pair-Net for Panoptic Scene Graph Generationcited by 17
  6. 2023Learning Situation Hyper-Graphs for Video Question Answeringcited by 15
  7. 2023LASER: A Neuro-Symbolic Framework for Learning Spatial-Temporal Scene Graphs with Weak Supervisioncited by 13
  8. 2024A Survey on Learning from Graphs with Heterophily: Recent Advances and Future Directionscited by 13
  9. 2024Text Guided Image Editing with Automatic Concept Locating and Forgettingcited by 12
  10. 2024FLAME: Frozen Large Language Models Enable Data-Efficient Language-Image Pre-trainingcited by 12
  11. 2023VisorGPT: Learning Visual Prior via Generative Pre-Trainingcited by 11
  12. 2024CODIS: Benchmarking Context-Dependent Visual Comprehension for Multimodal Large Language Modelscited by 11
38 more
  1. 2024Computer Vision for Primate Behavior Analysis in the Wildcited by 10
  2. 2024Towards Scene Graph Anticipationcited by 10
  3. 2023Haystack: A Panoptic Scene Graph Dataset to Evaluate Rare Predicate Classescited by 8
  4. 2024Multi-modal Generative AI: Multi-modal LLMs, Diffusions, and the Unificationcited by 8
  5. 2022Swinv2-Imagen: Hierarchical Vision Transformer Diffusion Models for Text-to-Image Generationcited by 7
  6. 2023Enhancing Scene Graph Generation with Hierarchical Relationships and Commonsense Knowledgecited by 7
  7. 2025RAVU: Retrieval Augmented Video Understanding with Compositional Reasoning over Graphcited by 7
  8. 2024Scene-Graph ViT: End-to-End Open-Vocabulary Visual Relationship Detectioncited by 6
  9. 2025ClusterViG: Efficient Globally Aware Vision GNNs via Image Partitioningcited by 6
  10. 2023Synthesizing Event-centric Knowledge Graphs of Daily Activities Using Virtual Spacecited by 5
  11. 2024From Engineering Diagrams to Graphs: Digitizing P&IDs with Transformerscited by 5
  12. 2025Agent Identity Evals: Measuring Agentic Identitycited by 5
  13. 2026RAG-3DSG: Enhancing 3D Scene Graphs with Re-Shot Guided Retrieval-Augmented Generationcited by 5
  14. 2022Visual Semantic Parsing: From Images to Abstract Meaning Representationcited by 4
  15. 2023Fine-Grained is Too Coarse: A Novel Data-Centric Approach for Efficient Scene Graph Generationcited by 4
  16. 2023CREPE: Learnable Prompting With CLIP Improves Visual Relationship Predictioncited by 4
  17. 2024Voxel Scene Graph for Intracranial Hemorrhagecited by 4
  18. 2025SAMJAM: Zero-Shot Video Scene Graph Generation for Egocentric Kitchen Videoscited by 4
  19. 20263D Scene Graphs: Open Challenges and Future Directionscited by 4
  20. 2023GKGNet: Group K-Nearest Neighbor based Graph Convolutional Network for Multi-Label Image Recognitioncited by 3
  21. 2024On the Topology Awareness and Generalization Performance of Graph Neural Networkscited by 3
  22. 2024BCTR: Bidirectional Conditioning Transformer for Scene Graph Generationcited by 3
  23. 2025CIVIL: Causal and Intuitive Visual Imitation Learningcited by 3
  24. 2025Generating Directed Graphs with Dual Attention and Asymmetric Encodingcited by 3
  25. 2025Decoding the Surgical Scene: A Scoping Review of Scene Graphs in Surgerycited by 3
  26. 2024A Review and Efficient Implementation of Scene Graph Generation Metricscited by 2
  27. 2024Effectively Leveraging CLIP for Generating Situational Summaries of Images and Videoscited by 2
  28. 2025FORGE-Tree: Diffusion-Forcing Tree Search for Long-Horizon Robot Manipulationcited by 2
  29. 2026The Semantic Lifecycle in Embodied AI: Acquisition, Representation and Storage via Foundation Modelscited by 2
  30. 2026NaLA: A 3D Native LLM Layout Agent for High-quality 3D Scene Generationcited by 2
  31. 2023Active Semantic Localization with Graph Neural Embeddingcited by 1
  32. 2024Beyond Visual Appearances: Privacy-sensitive Objects Identification via Hybrid Graph Reasoningcited by 1
  33. 2025Dynamic Scene Understanding from Vision-Language Representationscited by 1
  34. 2025FORESCENE: FOREcasting human activity via latent SCENE graphs diffusioncited by 1
  35. 2025SG-Tailor: Inter-Object Commonsense Relationship Reasoning for Scene Graph Manipulationcited by 1
  36. 2025FDSG: Forecasting Dynamic Scene Graphscited by 1
  37. 2025RecurGS: Interactive Scene Modeling via Discrete-State Recurrent Gaussian Fusioncited by 1
  38. 2026KG-ViP: Bridging Knowledge Grounding and Visual Perception in Multi-modal LLMs for Visual Question Answeringcited by 1

50 most cited of 73 All 73 (JSON)