# Vision Relation Transformer for Unbiased Scene Graph Generation

arXiv: https://arxiv.org/abs/2308.09472 · HTML: https://mlsanity.com/paper/2308.09472/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 55 references that are in the map.
- First posted on arXiv 18 Aug 2023.

## Builds on (most cited first, 30 of 50)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [Scene Graph Generation by Iterative Message Passing](https://mlsanity.com/paper/1701.02426/) (2017)
- [Neural Motifs: Scene Graph Parsing with Global Context](https://mlsanity.com/paper/1711.06640/) (2017)
- [Unbiased Scene Graph Generation from Biased Training](https://mlsanity.com/paper/2002.11949/) (2020)
- [Visual Relationship Detection with Language Priors](https://mlsanity.com/paper/1608.00187/) (2016)
- [Graph R-CNN for Scene Graph Generation](https://mlsanity.com/paper/1808.00191/) (2018)
- [Learning to Compose Dynamic Tree Structures for Visual Contexts](https://mlsanity.com/paper/1812.01880/) (2018)
- [Learning to Recover 3D Scene Shape from a Single Image](https://mlsanity.com/paper/2012.09365/) (2020)
- [A Comprehensive Survey of Deep Learning for Image Captioning](https://mlsanity.com/paper/1810.04020/) (2018)
- [GPS-Net: Graph Property Sensing Network for Scene Graph Generation](https://mlsanity.com/paper/2003.12962/) (2020)
- [Knowledge-Embedded Routing Network for Scene Graph Generation](https://mlsanity.com/paper/1903.03326/) (2019)
- [Scene Graph Generation from Objects, Phrases and Region Captions](https://mlsanity.com/paper/1707.09700/) (2017)
- [Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2104.00308/) (2021)
- [iCAN: Instance-Centric Attention Network for Human-Object Interaction Detection](https://mlsanity.com/paper/1808.10437/) (2018)
- [VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutions](https://mlsanity.com/paper/2003.05541/) (2020)
- [Visual Translation Embedding Network for Visual Relation Detection](https://mlsanity.com/paper/1702.08319/) (2017)
- [Energy-Based Learning for Scene Graph Generation](https://mlsanity.com/paper/2103.02221/) (2021)

## Built on by (most cited first, 18 of 18)

- [EGTR: Extracting Graph from Transformer for Scene Graph Generation](https://mlsanity.com/paper/2404.02072/) (2024)
- [Universal Scene Graph Generation](https://mlsanity.com/paper/2503.15005/) (2025)
- [Hydra-SGG: Hybrid Relation Assignment for One-stage Scene Graph Generation](https://mlsanity.com/paper/2409.10262/) (2024)
- [DeiSAM: Segment Anything with Deictic Prompting](https://mlsanity.com/paper/2402.14123/) (2024)
- [Reusing Attention for One-stage Lane Topology Understanding](https://mlsanity.com/paper/2507.17617/) (2025)
- [Leveraging Predicate and Triplet Learning for Scene Graph Generation](https://mlsanity.com/paper/2406.02038/) (2024)
- [Learning 4D Panoptic Scene Graph Generation from Rich 2D Visual Scene](https://mlsanity.com/paper/2503.15019/) (2025)
- [Scene Graph Generation Strategy with Co-occurrence Knowledge and Learnable Term Frequency](https://mlsanity.com/paper/2405.12648/) (2024)
- [Semantic Diversity-aware Prototype-based Learning for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2407.15396/) (2024)
- [DIAGen: Semantically Diverse Image Augmentation with Generative Models for Few-Shot Learning](https://mlsanity.com/paper/2408.14584/) (2024)
- [From Data to Modeling: Fully Open-vocabulary Scene Graph Generation](https://mlsanity.com/paper/2505.20106/) (2025)
- [A Reverse Causal Framework to Mitigate Spurious Correlations for Debiasing Scene Graph Generation](https://mlsanity.com/paper/2505.23451/) (2025)
- [Salience-SGG: Enhancing Unbiased Scene Graph Generation with Iterative Salience Estimation](https://mlsanity.com/paper/2601.08728/) (2026)
- [ALF: Adaptive Label Finetuning for Scene Graph Generation](https://mlsanity.com/paper/2312.17425/) (2023)
- [ART: Adaptive Relation Tuning for Generalized Relation Prediction](https://mlsanity.com/paper/2507.23543/) (2025)
- [Interaction-Centric Knowledge Infusion and Transfer for Open-Vocabulary Scene Graph Generation](https://mlsanity.com/paper/2511.05935/) (2025)
- [Modular Diffusion Models for Structured Visual Recognition](https://mlsanity.com/paper/2606.22702/) (2026)
- [Revisiting Scene Graph Generation from the Perspective of Detector-Conditioned Reachability](https://mlsanity.com/paper/2607.06176/) (2026)

## Abstract (arXiv)

Recent years have seen a growing interest in Scene Graph Generation (SGG), a comprehensive visual scene understanding task that aims to predict entity relationships using a relation encoder-decoder pipeline stacked on top of an object encoder-decoder backbone. Unfortunately, current SGG methods suffer from an information loss regarding the entities local-level cues during the relation encoding process. To mitigate this, we introduce the Vision rElation TransfOrmer (VETO), consisting of a novel local-level entity relation encoder. We further observe that many existing SGG methods claim to be unbiased, but are still biased towards either head or tail classes. To overcome this bias, we introduce a Mutually Exclusive ExperT (MEET) learning strategy that captures important relation features without bias towards head or tail classes. Experimental results on the VG and GQA datasets demonstrate that VETO + MEET boosts the predictive performance by up to 47 percentage over the state of the art while being 10 times smaller.