mlsanityData to 7 Oct 2026

SrTR: Self-reasoning Transformer with Visual-linguistic Knowledge for Scene Graph Generation

arXiv 2212.09329 ↗First posted 19 Dec 2022cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Objects in a scene are not always related. The execution efficiency of the one-stage scene graph generation approaches are quite high, which infer the effective relation between entity pairs using sparse proposal sets and a few queries. However, they only focus on the relation between subject and object in triplet set subject entity, predicate entity, object entity, ignoring the relation between subject and predicate or predicate and object, and the model lacks self-reasoning ability. In addition, linguistic modality has been neglected in the one-stage method. It is necessary to mine linguistic modality knowledge to improve model reasoning ability. To address the above-mentioned shortcomings, a Self-reasoning Transformer with Visual-linguistic Knowledge (SrTR) is proposed to add flexible self-reasoning ability to the model. An encoder-decoder architecture is adopted in SrTR, and a self-reasoning decoder is developed to complete three inferences of the triplet set, s+o-p, s+p-o and p+o-s. Inspired by the large-scale pre-training image-text foundation models, visual-linguistic prior knowledge is introduced and a visual-linguistic alignment strategy is designed to project visual representations into semantic spaces with prior knowledge to aid relational reasoning. Experiments on the Visual Genome dataset demonstrate the superiority and fast inference ability of the proposed method.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 30 references that are in the map. First posted on arXiv 19 Dec 2022.

Builds on

27 of 30 references.

References · 27

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2017Scene Graph Generation by Iterative Message Passingcited by 462
  5. 2017Neural Motifs: Scene Graph Parsing with Global Contextcited by 413
  6. 2020Unbiased Scene Graph Generation from Biased Trainingcited by 355
  7. 2018Image Generation from Scene Graphscited by 324
  8. 2018Graph R-CNN for Scene Graph Generationcited by 296
  9. 2018Auto-Encoding Scene Graphs for Image Captioningcited by 254
  10. 2020GPS-Net: Graph Property Sensing Network for Scene Graph Generationcited by 165
  11. 2019Knowledge-Embedded Routing Network for Scene Graph Generationcited by 163
  12. 2021Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generationcited by 137
15 more
  1. 2017Inferring and Executing Programs for Visual Reasoningcited by 130
  2. 2020Empowering Things with Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Thingscited by 110
  3. 2021Energy-Based Learning for Scene Graph Generationcited by 104
  4. 2018Explainable and Explicit Visual Reasoning over Scene Graphscited by 103
  5. 2019Graphical Contrastive Losses for Scene Graph Parsingcited by 100
  6. 2022RelTR: Relation Transformer for Scene Graph Generationcited by 95
  7. 2020Semantic Image Manipulation Using Scene Graphscited by 82
  8. 2022Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph Generationcited by 67
  9. 2021Learning of Visual Relations: The Devil is in the Tailscited by 65
  10. 2021Fully Convolutional Scene Graph Generationcited by 57
  11. 2021Structured Sparse R-CNN for Direct Scene Graph Generationcited by 44
  12. 2022RU-Net: Regularized Unrolling Network for Scene Graph Generationcited by 28
  13. 2020Relation Transformer Networkcited by 20
  14. 2021Understanding the Role of Scene Graphs in Visual Question Answeringcited by 20
  15. 2020Tackling the Unannotated: Scene Graph Generation with Bias-Reduced Modelscited by 19

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SGR3 Model: Scene Graph Retrieval-Reasoning Model in 3D
  2. 2026CAPruner: Conceptual-Adjacent Scene Graph Pruner for Enhancing 3D Spatial Reasoning of Large Language Models
  3. 2025Generative Visual Commonsense Answering and Explaining with Generative Scene Graph Constructing
  4. 2026Scene Graph Thinking: Reinforcing Structured Visual Reasoning for Multimodal Large Language Models
  5. 2024SceneLLM: Implicit Language Reasoning in LLM for Dynamic Scene Graph Generation
  6. 2026Visual Commonsense Driven Knowledge Refinements for Scene Graph Generation
  7. 2025SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards
  8. 2025VOST-SGG: VLM-Aided One-Stage Spatio-Temporal Scene Graph Generation