# Unbiased Scene Graph Generation by Type-Aware Message Passing on Heterogeneous and Dual Graphs

arXiv: https://arxiv.org/abs/2411.13287 · HTML: https://mlsanity.com/paper/2411.13287/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 20 Nov 2024.

## Builds on (most cited first, 24 of 24)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Scene Graph Generation by Iterative Message Passing](https://mlsanity.com/paper/1701.02426/) (2017)
- [Neural Motifs: Scene Graph Parsing with Global Context](https://mlsanity.com/paper/1711.06640/) (2017)
- [Learning to Compose Dynamic Tree Structures for Visual Contexts](https://mlsanity.com/paper/1812.01880/) (2018)
- [GPS-Net: Graph Property Sensing Network for Scene Graph Generation](https://mlsanity.com/paper/2003.12962/) (2020)
- [Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2104.00308/) (2021)
- [Bridging Knowledge Graphs to Generate Scene Graphs](https://mlsanity.com/paper/2001.02314/) (2020)
- [Scene Graph Generation: A Comprehensive Survey](https://mlsanity.com/paper/2201.00443/) (2022)
- [Recovering the Unbiased Scene Graphs from the Biased Ones](https://mlsanity.com/paper/2107.02112/) (2021)
- [Learning to Generate Scene Graph from Natural Language Supervision](https://mlsanity.com/paper/2109.02227/) (2021)
- [Prototype-based Embedding Network for Scene Graph Generation](https://mlsanity.com/paper/2303.07096/) (2023)
- [RU-Net: Regularized Unrolling Network for Scene Graph Generation](https://mlsanity.com/paper/2205.01297/) (2022)
- [HL-Net: Heterophily Learning Network for Scene Graph Generation](https://mlsanity.com/paper/2205.01316/) (2022)
- [Unbiased Heterogeneous Scene Graph Generation with Relation-aware Message Passing Neural Network](https://mlsanity.com/paper/2212.00443/) (2022)
- [Label Semantic Knowledge Distillation for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2208.03763/) (2022)
- [Dual-branch Hybrid Learning Network for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2207.07913/) (2022)
- [HOSE-Net: Higher Order Structure Embedded Network for Scene Graph Generation](https://mlsanity.com/paper/2008.05156/) (2020)
- [Neural Belief Propagation for Scene Graph Generation](https://mlsanity.com/paper/2112.05727/) (2021)
- [Importance Weighted Structure Learning for Scene Graph Generation](https://mlsanity.com/paper/2205.07017/) (2022)
- [Peer Learning for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2301.00146/) (2022)
- [Constrained Structure Learning for Scene Graph Generation](https://mlsanity.com/paper/2201.11697/) (2022)
- [Semantic Scene Graph Generation Based on an Edge Dual Scene Graph and Message Passing Neural Network](https://mlsanity.com/paper/2311.01192/) (2023)

## Abstract (arXiv)

Although great progress has been made in the research of unbiased scene graph generation, issues still hinder improving the predictive performance of both head and tail classes. An unbiased scene graph generation (TA-HDG) is proposed to address these issues. For modeling interactive and non-interactive relations, the Interactive Graph Construction is proposed to model the dependence of relations on objects by combining heterogeneous and dual graph, when modeling relations between multiple objects. It also implements a subject-object pair selection strategy to reduce meaningless edges. Moreover, the Type-Aware Message Passing enhances the understanding of complex interactions by capturing intra- and inter-type context in the Intra-Type and Inter-Type stages. The Intra-Type stage captures the semantic context of inter-relaitons and inter-objects. On this basis, the Inter-Type stage captures the context between objects and relations for interactive and non-interactive relations, respectively. Experiments on two datasets show that TA-HDG achieves improvements in the metrics of R@K and mR@K, which proves that TA-HDG can accurately predict the tail class while maintaining the competitive performance of the head class.