mlsanityData to 7 Oct 2026

Vision Relation Transformer for Unbiased Scene Graph Generation

arXiv 2308.09472 ↗First posted 18 Aug 2023cs.CV cs.AIcited by 18

Not on the map. Why?

Abstract (arXiv)

Recent years have seen a growing interest in Scene Graph Generation (SGG), a comprehensive visual scene understanding task that aims to predict entity relationships using a relation encoder-decoder pipeline stacked on top of an object encoder-decoder backbone. Unfortunately, current SGG methods suffer from an information loss regarding the entities local-level cues during the relation encoding process. To mitigate this, we introduce the Vision rElation TransfOrmer (VETO), consisting of a novel local-level entity relation encoder. We further observe that many existing SGG methods claim to be unbiased, but are still biased towards either head or tail classes. To overcome this bias, we introduce a Mutually Exclusive ExperT (MEET) learning strategy that captures important relation features without bias towards head or tail classes. Experimental results on the VG and GQA datasets demonstrate that VETO + MEET boosts the predictive performance by up to 47 percentage over the state of the art while being 10 times smaller.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 50 of its 55 references that are in the map. First posted on arXiv 18 Aug 2023.

Builds on

50 of 55 references.

References · 50

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2017Graph Attention Networkscited by 8,854
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2017Mask R-CNNcited by 7,259
  9. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  10. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  11. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  12. 2015VQA: Visual Question Answeringcited by 2,489
38 more
  1. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  2. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  3. 2017Scene Graph Generation by Iterative Message Passingcited by 462
  4. 2017Neural Motifs: Scene Graph Parsing with Global Contextcited by 413
  5. 2020Unbiased Scene Graph Generation from Biased Trainingcited by 355
  6. 2016Visual Relationship Detection with Language Priorscited by 333
  7. 2018Graph R-CNN for Scene Graph Generationcited by 296
  8. 2018Learning to Compose Dynamic Tree Structures for Visual Contextscited by 258
  9. 2020Learning to Recover 3D Scene Shape from a Single Imagecited by 214
  10. 2018A Comprehensive Survey of Deep Learning for Image Captioningcited by 213
  11. 2020GPS-Net: Graph Property Sensing Network for Scene Graph Generationcited by 165
  12. 2019Knowledge-Embedded Routing Network for Scene Graph Generationcited by 163
  13. 2017Scene Graph Generation from Objects, Phrases and Region Captionscited by 154
  14. 2021Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generationcited by 137
  15. 2018iCAN: Instance-Centric Attention Network for Human-Object Interaction Detectioncited by 126
  16. 2020VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutionscited by 115
  17. 2017Visual Translation Embedding Network for Visual Relation Detectioncited by 112
  18. 2021Energy-Based Learning for Scene Graph Generationcited by 104
  19. 2018Explainable and Explicit Visual Reasoning over Scene Graphscited by 103
  20. 2019Scene Graph Generation with External Knowledge and Image Reconstructioncited by 102
  21. 2017Detecting Visual Relationships with Deep Relational Networkscited by 97
  22. 2020Bridging Knowledge Graphs to Generate Scene Graphscited by 86
  23. 2020PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph Generationcited by 75
  24. 2021Recovering the Unbiased Scene Graphs from the Biased Onescited by 71
  25. 2022Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph Generationcited by 67
  26. 2021Learning of Visual Relations: The Devil is in the Tailscited by 65
  27. 2020CogTree: Cognition Tree Loss for Unbiased Scene Graph Generationcited by 62
  28. 2017Visual Relationship Detection with Internal and External Linguistic Knowledge Distillationcited by 60
  29. 2022Fine-Grained Scene Graph Generation with Data Transfercited by 57
  30. 2018LinkNet: Relational Embedding for Scene Graphcited by 45
  31. 2018Zoom-Net: Mining Deep Feature Interactions for Visual Relationship Recognitioncited by 41
  32. 2018Referring Relationshipscited by 29
  33. 2020Relation Transformer Networkcited by 20
  34. 2020Tackling the Unannotated: Scene Graph Generation with Bias-Reduced Modelscited by 19
  35. 2017Natural Language Guided Visual Relationship Detectioncited by 15
  36. 2019Improving Visual Relation Detection using Depth Mapscited by 8
  37. 2021GTNet:Guided Transformer Network for Detecting Human-Object Interactionscited by 8
  38. 2018Visual Relationship Prediction via Label Clustering and Incorporation of Depth Informationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 7; 2025: 7; 2026: 3.

18 papers.

Sort:

By area:

  1. 2024EGTR: Extracting Graph from Transformer for Scene Graph Generationcited by 45
  2. 2025Universal Scene Graph Generationcited by 11
  3. 2024Hydra-SGG: Hybrid Relation Assignment for One-stage Scene Graph Generationcited by 10
  4. 2024DeiSAM: Segment Anything with Deictic Promptingcited by 9
  5. 2025Reusing Attention for One-stage Lane Topology Understandingcited by 9
  6. 2024Leveraging Predicate and Triplet Learning for Scene Graph Generationcited by 8
  7. 2025Learning 4D Panoptic Scene Graph Generation from Rich 2D Visual Scenecited by 6
  8. 2024Scene Graph Generation Strategy with Co-occurrence Knowledge and Learnable Term Frequencycited by 4
  9. 2024Semantic Diversity-aware Prototype-based Learning for Unbiased Scene Graph Generationcited by 4
  10. 2024DIAGen: Semantically Diverse Image Augmentation with Generative Models for Few-Shot Learningcited by 3
  11. 2025From Data to Modeling: Fully Open-vocabulary Scene Graph Generationcited by 2
  12. 2025A Reverse Causal Framework to Mitigate Spurious Correlations for Debiasing Scene Graph Generationcited by 1
6 more
  1. 2026Salience-SGG: Enhancing Unbiased Scene Graph Generation with Iterative Salience Estimationcited by 1
  2. 2023ALF: Adaptive Label Finetuning for Scene Graph Generationcited by 0
  3. 2025ART: Adaptive Relation Tuning for Generalized Relation Predictioncited by 0
  4. 2025Interaction-Centric Knowledge Infusion and Transfer for Open-Vocabulary Scene Graph Generationcited by 0
  5. 2026Modular Diffusion Models for Structured Visual Recognitioncited by 0
  6. 2026Revisiting Scene Graph Generation from the Perspective of Detector-Conditioned Reachabilitycited by 0