# Unbiased Scene Graph Generation in Videos

arXiv: https://arxiv.org/abs/2304.00733 · HTML: https://mlsanity.com/paper/2304.00733/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 68 references that are in the map.
- First posted on arXiv 3 Apr 2023.

## Builds on (most cited first, 30 of 45)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [VideoBERT: A Joint Model for Video and Language Representation Learning](https://mlsanity.com/paper/1904.01766/) (2019)
- [A Survey on Visual Transformer](https://mlsanity.com/paper/2012.12556/) (2020)
- [Neural Turing Machines](https://mlsanity.com/paper/1410.5401/) (2014)
- [Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding](https://mlsanity.com/paper/1604.01753/) (2016)
- [Neural Motifs: Scene Graph Parsing with Global Context](https://mlsanity.com/paper/1711.06640/) (2017)
- [Dynamic Few-Shot Visual Learning without Forgetting](https://mlsanity.com/paper/1804.09458/) (2018)
- [Unbiased Scene Graph Generation from Biased Training](https://mlsanity.com/paper/2002.11949/) (2020)
- [Visual Relationship Detection with Language Priors](https://mlsanity.com/paper/1608.00187/) (2016)
- [Learning to Compose Dynamic Tree Structures for Visual Contexts](https://mlsanity.com/paper/1812.01880/) (2018)
- [Action Genome: Actions as Composition of Spatio-temporal Scene Graphs](https://mlsanity.com/paper/1912.06992/) (2019)
- [GPS-Net: Graph Property Sensing Network for Scene Graph Generation](https://mlsanity.com/paper/2003.12962/) (2020)
- [Knowledge-Embedded Routing Network for Scene Graph Generation](https://mlsanity.com/paper/1903.03326/) (2019)
- [Scene Graph Generation from Objects, Phrases and Region Captions](https://mlsanity.com/paper/1707.09700/) (2017)
- [Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2104.00308/) (2021)
- [Estimating and Exploiting the Aleatoric Uncertainty in Surface Normal Estimation](https://mlsanity.com/paper/2109.09881/) (2021)
- [SGTR: End-to-end Scene Graph Generation with Transformer](https://mlsanity.com/paper/2112.12970/) (2021)

## Built on by (most cited first, 30 of 39)

- [Action Scene Graphs for Long-Form Understanding of Egocentric Videos](https://mlsanity.com/paper/2312.03391/) (2023)
- [OED: Towards One-stage End-to-End Dynamic Scene Graph Generation](https://mlsanity.com/paper/2405.16925/) (2024)
- [HIG: Hierarchical Interlacement Graph Approach to Scene Graph Generation in Video Understanding](https://mlsanity.com/paper/2312.03050/) (2023)
- [HyperGLM: HyperGraph for Video Scene Graph Generation and Anticipation](https://mlsanity.com/paper/2411.18042/) (2024)
- [Constructing Holistic Spatio-Temporal Scene Graph for Video Semantic Role Labeling](https://mlsanity.com/paper/2308.05081/) (2023)
- [LASER: A Neuro-Symbolic Framework for Learning Spatial-Temporal Scene Graphs with Weak Supervision](https://mlsanity.com/paper/2304.07647/) (2023)
- [CYCLO: Cyclic Graph Transformer Approach to Multi-Object Relationship Modeling in Aerial Videos](https://mlsanity.com/paper/2406.01029/) (2024)
- [DIFFVSGG: Diffusion-Driven Online Video Scene Graph Generation](https://mlsanity.com/paper/2503.13957/) (2025)
- [Universal Scene Graph Generation](https://mlsanity.com/paper/2503.15005/) (2025)
- [Towards Scene Graph Anticipation](https://mlsanity.com/paper/2403.04899/) (2024)
- [Motion-aware Contrastive Learning for Temporal Panoptic Scene Graph Generation](https://mlsanity.com/paper/2412.07160/) (2024)
- [Conformal Prediction and MLLM aided Uncertainty Quantification in Scene Graph Generation](https://mlsanity.com/paper/2503.13947/) (2025)
- [FloCoDe: Unbiased Dynamic Scene Graph Generation with Temporal Consistency and Correlation Debiasing](https://mlsanity.com/paper/2310.16073/) (2023)
- [Towards Unbiased and Robust Spatio-Temporal Scene Graph Generation and Anticipation](https://mlsanity.com/paper/2411.13059/) (2024)
- [Unbiased Video Scene Graph Generation via Visual and Semantic Dual Debiasing](https://mlsanity.com/paper/2503.00548/) (2025)
- [SceneLLM: Implicit Language Reasoning in LLM for Dynamic Scene Graph Generation](https://mlsanity.com/paper/2412.11026/) (2024)
- [GraphThinker: Reinforcing Temporally Grounded Video Reasoning with Event Graph Thinking](https://mlsanity.com/paper/2602.17555/) (2026)
- [Improving Scene Graph Generation with Superpixel-Based Interaction Learning](https://mlsanity.com/paper/2308.02339/) (2023)
- [Situational Scene Graph for Structured Human-centric Situation Understanding](https://mlsanity.com/paper/2410.22829/) (2024)
- [Weakly Supervised Video Scene Graph Generation via Natural Language Supervision](https://mlsanity.com/paper/2502.15370/) (2025)
- [Synthetic Visual Genome 2: Extracting Large-scale Spatio-Temporal Scene Graphs from Videos](https://mlsanity.com/paper/2602.23543/) (2026)
- [EgoGraph: Temporal Knowledge Graph for Egocentric Video Understanding](https://mlsanity.com/paper/2602.23709/) (2026)
- [UNO: Unifying One-stage Video Scene Graph Generation via Object-Centric Visual Representation Learning](https://mlsanity.com/paper/2509.06165/) (2025)
- [IMPACT-CYCLE: A Contract-Based Multi-Agent System for Claim-Level Supervisory Correction of Long-Video Semantic Memory](https://mlsanity.com/paper/2604.20136/) (2026)
- [TD^2-Net: Toward Denoising and Debiasing for Dynamic Scene Graph Generation](https://mlsanity.com/paper/2401.12479/) (2024)
- [FORESCENE: FOREcasting human activity via latent SCENE graphs diffusion](https://mlsanity.com/paper/2503.06182/) (2025)
- [FDSG: Forecasting Dynamic Scene Graphs](https://mlsanity.com/paper/2506.01487/) (2025)
- [VOST-SGG: VLM-Aided One-Stage Spatio-Temporal Scene Graph Generation](https://mlsanity.com/paper/2512.05524/) (2025)
- [Revisiting Weakly-Supervised Video Scene Graph Generation via Pair Affinity Learning](https://mlsanity.com/paper/2603.21559/) (2026)
- [SceneGraphVLM: Dynamic Scene Graph Generation from Video with Vision-Language Models](https://mlsanity.com/paper/2605.13667/) (2026)

## Abstract (arXiv)

The task of dynamic scene graph generation (SGG) from videos is complicated and challenging due to the inherent dynamics of a scene, temporal fluctuation of model predictions, and the long-tailed distribution of the visual relationships in addition to the already existing challenges in image-based SGG. Existing methods for dynamic SGG have primarily focused on capturing spatio-temporal context using complex architectures without addressing the challenges mentioned above, especially the long-tailed distribution of relationships. This often leads to the generation of biased scene graphs. To address these challenges, we introduce a new framework called TEMPURA: TEmporal consistency and Memory Prototype guided UnceRtainty Attenuation for unbiased dynamic SGG. TEMPURA employs object-level temporal consistencies via transformer-based sequence modeling, learns to synthesize unbiased relationship representations using memory-guided training, and attenuates the predictive uncertainty of visual relations using a Gaussian Mixture Model (GMM). Extensive experiments demonstrate that our method achieves significant (up to 10% in some cases) performance gain over existing methods highlighting its superiority in generating more unbiased scene graphs.