mlsanityData to 7 Oct 2026

RelTR: Relation Transformer for Scene Graph Generation

arXiv 2201.11460 ↗First posted 27 Jan 2022cs.CVcited by 95

Not on the map. Why?

Abstract (arXiv)

Different objects in the same scene are more or less related to each other, but only a limited number of these relationships are noteworthy. Inspired by DETR, which excels in object detection, we view scene graph generation as a set prediction problem and propose an end-to-end scene graph generation model RelTR which has an encoder-decoder architecture. The encoder reasons about the visual feature context while the decoder infers a fixed-size set of triplets subject-predicate-object using different types of attention mechanisms with coupled subject and object queries. We design a set prediction loss performing the matching between the ground truth and predicted triplets for the end-to-end training. In contrast to most existing scene graph generation methods, RelTR is a one-stage method that predicts a set of relationships directly only using visual appearance without combining entities and labeling all possible predicates. Extensive experiments on the Visual Genome and Open Images V6 datasets demonstrate the superior performance and fast inference of our model.

Key facts. Cited by 95 papers in the map (data through 7 Oct 2026). Builds on 75 of its 88 references that are in the map. First posted on arXiv 27 Jan 2022.

Builds on

75 of 88 references.

References · 75

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  6. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  7. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
  8. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
  9. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  10. 2019Objects as Pointscited by 1,172
  11. 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
  12. 2020Long-tail learning via logit adjustmentcited by 605
38 more
  1. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 542
  2. 2017Scene Graph Generation by Iterative Message Passingcited by 462
  3. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  4. 2017Neural Motifs: Scene Graph Parsing with Global Contextcited by 413
  5. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 379
  6. 2020Unbiased Scene Graph Generation from Biased Trainingcited by 355
  7. 2016Visual Relationship Detection with Language Priorscited by 333
  8. 2018Image Generation from Scene Graphscited by 324
  9. 2018Graph R-CNN for Scene Graph Generationcited by 296
  10. 2018Learning to Compose Dynamic Tree Structures for Visual Contextscited by 258
  11. 2018Auto-Encoding Scene Graphs for Image Captioningcited by 254
  12. 2019Action Genome: Actions as Composition of Spatio-temporal Scene Graphscited by 249
  13. 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
  14. 2020GPS-Net: Graph Property Sensing Network for Scene Graph Generationcited by 165
  15. 2019Knowledge-Embedded Routing Network for Scene Graph Generationcited by 163
  16. 2017Scene Graph Generation from Objects, Phrases and Region Captionscited by 154
  17. 2015End-to-end people detection in crowded scenescited by 140
  18. 2018Character-Level Language Modeling with Deeper Self-Attentioncited by 138
  19. 2021Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generationcited by 137
  20. 2021End-to-End Human Object Interaction Detection with HOI Transformercited by 134
  21. 2017Inferring and Executing Programs for Visual Reasoningcited by 130
  22. 2021Efficient DETR: Improving End-to-End Object Detector with Dense Priorcited by 128
  23. 2017Visual Translation Embedding Network for Visual Relation Detectioncited by 112
  24. 2021Energy-Based Learning for Scene Graph Generationcited by 104
  25. 2018Explainable and Explicit Visual Reasoning over Scene Graphscited by 103
  26. 2019Scene Graph Generation with External Knowledge and Image Reconstructioncited by 102
  27. 2021SGTR: End-to-end Scene Graph Generation with Transformercited by 102
  28. 2019Graphical Contrastive Losses for Scene Graph Parsingcited by 100
  29. 2017Detecting Visual Relationships with Deep Relational Networkscited by 97
  30. 2021Spatial-Temporal Transformer for Dynamic Scene Graph Generationcited by 95
  31. 2020Bridging Knowledge Graphs to Generate Scene Graphscited by 86
  32. 2019Unpaired Image Captioning via Scene Graph Alignmentscited by 82
  33. 2018Factorizable Net: An Efficient Subgraph-based Framework for Scene Graph Generationcited by 78
  34. 2019Specifying Object Attributes and Relations in Interactive Scene Generationcited by 76
  35. 2020PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph Generationcited by 75
  36. 2018Attentive Relational Networks for Mapping Images to Scene Graphscited by 72
  37. 2021Recovering the Unbiased Scene Graphs from the Biased Onescited by 71
  38. 2021Learning of Visual Relations: The Devil is in the Tailscited by 65

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 23; 2024: 27; 2025: 21; 2026: 14.

95 papers.

Sort:

By area:

  1. 2024The All-Seeing Project V2: Towards General Relation Comprehension of the Open Worldcited by 90
  2. 2022Relationformer: A Unified Framework for Image-to-Graph Generationcited by 67
  3. 2022RLIP: Relational Language-Image Pre-training for Human-Object Interaction Detectioncited by 54
  4. 2023FACTUAL: A Benchmark for Faithful and Consistent Textual Scene Graph Parsingcited by 50
  5. 2023Panoptic Video Scene Graph Generationcited by 46
  6. 2024EGTR: Extracting Graph from Transformer for Scene Graph Generationcited by 45
  7. 2023RLIPv2: Fast Scaling of Relational Language-Image Pre-trainingcited by 44
  8. 2024From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Modelscited by 43
  9. 2023CLIP-VG: Self-paced Curriculum Adapting of CLIP for Visual Groundingcited by 35
  10. 2024T2SG: Traffic Topology Scene Graph for Topology Reasoning in Autonomous Drivingcited by 35
  11. 2024Graph Transformers: A Surveycited by 32
  12. 2024Segment Anything Model for Road Network Graph Extractioncited by 28
38 more
  1. 2022Iterative Scene Graph Generationcited by 24
  2. 2022Iterative Scene Graph Generation with Generative Transformerscited by 22
  3. 20244D Panoptic Scene Graph Generationcited by 22
  4. 2024OED: Towards One-stage End-to-End Dynamic Scene Graph Generationcited by 22
  5. 2023HiLo: Exploiting High Low Frequency Relations for Unbiased Panoptic Scene Graph Generationcited by 19
  6. 2024HyperGLM: HyperGraph for Video Scene Graph Generation and Anticipationcited by 17
  7. 2024Scene Graph Generation with Role-Playing Large Language Modelscited by 16
  8. 2023Towards a Unified Transformer-based Framework for Scene Graph Generation and Human-object Interaction Detectioncited by 15
  9. 2022Adaptive Fine-Grained Predicates Learning for Scene Graph Generationcited by 14
  10. 2023Unified Visual Relationship Detection with Vision and Language Modelscited by 13
  11. 2023VLPrompt: Vision-Language Prompting for Panoptic Scene Graph Generationcited by 12
  12. 2025Universal Scene Graph Generationcited by 11
  13. 2025Compile Scene Graphs with Reinforcement Learningcited by 11
  14. 2023Spatial-Temporal Knowledge-Embedded Transformer for Video Scene Graph Generationcited by 10
  15. 2024Enhancing Vision-Language Models with Scene Graphs for Traffic Accident Understandingcited by 10
  16. 2024Hydra-SGG: Hybrid Relation Assignment for One-stage Scene Graph Generationcited by 10
  17. 2025DIFFVSGG: Diffusion-Driven Online Video Scene Graph Generationcited by 10
  18. 2023HyperSparse Neural Networks: Shifting Exploration to Exploitation through Adaptive Regularizationcited by 9
  19. 2025Reusing Attention for One-stage Lane Topology Understandingcited by 9
  20. 2022SSGVS: Semantic Scene Graph-to-Video Synthesiscited by 8
  21. 2022Probabilistic Debiasing of Scene Graphscited by 8
  22. 2023Choose your Data Wisely: A Framework for Semantic Counterfactualscited by 7
  23. 2023Single-Stage Visual Relationship Learning using Conditional Queriescited by 7
  24. 2023Enhancing Scene Graph Generation with Hierarchical Relationships and Commonsense Knowledgecited by 7
  25. 2025Open World Scene Graph Generation using Vision Language Modelscited by 7
  26. 2025Abstract 3D Perception for Spatial Intelligence in Vision-Language Modelscited by 7
  27. 2024Scene-Graph ViT: End-to-End Open-Vocabulary Visual Relationship Detectioncited by 6
  28. 2024Tri-modal Confluence with Temporal Dynamics for Scene Graph Generation in Operating Roomscited by 6
  29. 2024DAVINCI: A Single-Stage Architecture for Constrained CAD Sketch Inferencecited by 6
  30. 2025Unbiased Video Scene Graph Generation via Visual and Semantic Dual Debiasingcited by 6
  31. 2025Learning 4D Panoptic Scene Graph Generation from Rich 2D Visual Scenecited by 6
  32. 2023Skew Class-balanced Re-weighting for Unbiased Scene Graph Generationcited by 5
  33. 2024Neural Assembler: Learning to Generate Fine-Grained Robotic Assembly Instructions from Multi-View Imagescited by 5
  34. 2024SceneLLM: Implicit Language Reasoning in LLM for Dynamic Scene Graph Generationcited by 5
  35. 2022Edge Computing for Semantic Communication Enabled Metaverse: An Incentive Mechanism Designcited by 4
  36. 2023Fine-Grained is Too Coarse: A Novel Data-Centric Approach for Efficient Scene Graph Generationcited by 4
  37. 2024Situational Scene Graph for Structured Human-centric Situation Understandingcited by 4
  38. 2025DyGEnc: Encoding a Sequence of Textual Scene Graphs to Reason and Answer Questions in Dynamic Scenescited by 4

50 most cited of 95 All 95 (JSON)