mlsanityData to 7 Oct 2026

Scene Graph Generation by Iterative Message Passing

arXiv 1701.02426 ↗First posted 10 Jan 2017cs.CVcited by 462

Not on the map. Why?

Cited by 38 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Understanding a visual scene goes beyond recognizing individual objects in isolation. Relationships between objects also constitute rich semantic information about the scene. In this work, we explicitly model the objects and their relationships using scene graphs, a visually-grounded graphical structure of an image. We propose a novel end-to-end model that generates such structured scene representation from an input image. The model solves the scene graph inference problem using standard RNNs and learns to iteratively improves its predictions via message passing. Our joint inference model can take advantage of contextual cues to make better predictions on objects and their relationships. The experiments show that our model significantly outperforms previous methods for generating scene graphs using Visual Genome dataset and inferring support relations with NYU Depth v2 dataset.

Key facts. Cited by 462 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Jan 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 86; 2022: 88; 2023: 102; 2024: 78; 2025: 60; 2026: 48.

462 papers.

Sort:

By area:

  1. 2021Grounded Language-Image Pre-trainingcited by 1,147
  2. 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Modelscited by 242
  3. 2022Vision GNN: An Image is Worth Graph of Nodescited by 185
  4. 2021CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Modelscited by 182
  5. 2023Compositional Chain-of-Thought Prompting for Large Multimodal Modelscited by 160
  6. 2021SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequencescited by 159
  7. 2021Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphscited by 144
  8. 2021Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generationcited by 137
  9. 2021A Comprehensive Survey of Scene Graphs: Generation and Applicationcited by 125
  10. 2021Weisfeiler and Leman go Machine Learning: The Story so farcited by 121
  11. 2022Panoptic Scene Graph Generationcited by 114
  12. 2021Energy-Based Learning for Scene Graph Generationcited by 104
38 more
  1. 2021SGTR: End-to-end Scene Graph Generation with Transformercited by 102
  2. 2024SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understandingcited by 101
  3. 2021Spatial-Temporal Transformer for Dynamic Scene Graph Generationcited by 95
  4. 2022RelTR: Relation Transformer for Scene Graph Generationcited by 95
  5. 2021ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentationcited by 93
  6. 2024The All-Seeing Project V2: Towards General Relation Comprehension of the Open Worldcited by 90
  7. 2021Chest ImaGenome Dataset for Clinical Reasoningcited by 89
  8. 2024LLARVA: Vision-Action Instruction Tuning Enhances Robot Learningcited by 87
  9. 2021Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphscited by 83
  10. 2023ReVersion: Diffusion-Based Relation Inversion from Imagescited by 82
  11. 2024Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationshipscited by 76
  12. 2022Multi-Modal Knowledge Graph Construction and Application: A Surveycited by 74
  13. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  14. 2021Graph-MLP: Node Classification without Message Passing in Graphcited by 73
  15. 2022Scene Graph Generation: A Comprehensive Surveycited by 73
  16. 2023Dense and Aligned Captions (DAC) Promote Compositional Reasoning in VL Modelscited by 72
  17. 2021Recovering the Unbiased Scene Graphs from the Biased Onescited by 71
  18. 2022Teaching Structured Vision&Language Concepts to Vision&Language Modelscited by 71
  19. 2022Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph Generationcited by 67
  20. 2022Relationformer: A Unified Framework for Image-to-Graph Generationcited by 67
  21. 2021Learning of Visual Relations: The Devil is in the Tailscited by 65
  22. 2021Learning to Generate Scene Graph from Natural Language Supervisioncited by 64
  23. 2022The Devil is in the Labels: Noisy Label Correction for Robust Scene Graph Generationcited by 60
  24. 2021ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoningcited by 58
  25. 2021Fully Convolutional Scene Graph Generationcited by 57
  26. 2022Fine-Grained Scene Graph Generation with Data Transfercited by 57
  27. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  28. 2022Towards Open-vocabulary Scene Graph Generation with Prompt-based Finetuningcited by 54
  29. 2025Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spacescited by 53
  30. 2021Target Adaptive Context Aggregation for Video Scene Graph Generationcited by 51
  31. 2021From General to Specific: Informative Scene Graph Generation via Balance Adjustmentcited by 51
  32. 2021Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networkscited by 50
  33. 2023FACTUAL: A Benchmark for Faithful and Consistent Textual Scene Graph Parsingcited by 50
  34. 2024RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulationcited by 48
  35. 2021Exploiting Edge-Oriented Reasoning for 3D Point-based Scene Graph Analysiscited by 47
  36. 2023Panoptic Video Scene Graph Generationcited by 46
  37. 2022Graph Inverse Reinforcement Learning from Diverse Videoscited by 45
  38. 2024EGTR: Extracting Graph from Transformer for Scene Graph Generationcited by 45

50 most cited of 462 All 462 (JSON)