# An Empirical Study on Leveraging Scene Graphs for Visual Question Answering

arXiv: https://arxiv.org/abs/1907.12133 · HTML: https://mlsanity.com/paper/1907.12133/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jul 2019.

## Built on by (most cited first, 21 of 21)

- [A Comprehensive Survey of Scene Graphs: Generation and Application](https://mlsanity.com/paper/2104.01111/) (2021)
- [Scene Graph Generation: A Comprehensive Survey](https://mlsanity.com/paper/2201.00443/) (2022)
- [Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2203.09811/) (2022)
- [EGTR: Extracting Graph from Transformer for Scene Graph Generation](https://mlsanity.com/paper/2404.02072/) (2024)
- [Structure-CLIP: Towards Scene Graph Knowledge to Enhance Multi-modal Structured Representations](https://mlsanity.com/paper/2305.06152/) (2023)
- [Probabilistic Modeling of Semantic Ambiguity for Scene Graph Generation](https://mlsanity.com/paper/2103.05271/) (2021)
- [BGT-Net: Bidirectional GRU Transformer Network for Scene Graph Generation](https://mlsanity.com/paper/2109.05346/) (2021)
- [Unbiased Heterogeneous Scene Graph Generation with Relation-aware Message Passing Neural Network](https://mlsanity.com/paper/2212.00443/) (2022)
- [Pair then Relation: Pair-Net for Panoptic Scene Graph Generation](https://mlsanity.com/paper/2307.08699/) (2023)
- [Discovering the Unknown Knowns: Turning Implicit Knowledge in the Dataset into Explicit Training Examples for Visual Question Answering](https://mlsanity.com/paper/2109.06122/) (2021)
- [HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generation](https://mlsanity.com/paper/2403.12033/) (2024)
- [Joint learning of object graph and relation graph for visual question answering](https://mlsanity.com/paper/2205.04188/) (2022)
- [Graph Neural Networks in Vision-Language Image Understanding: A Survey](https://mlsanity.com/paper/2303.03761/) (2023)
- [Enhancing Vision-Language Models with Scene Graphs for Traffic Accident Understanding](https://mlsanity.com/paper/2407.05910/) (2024)
- [VeriGraph: Scene Graphs for Execution Verifiable Robot Planning](https://mlsanity.com/paper/2411.10446/) (2024)
- [From Vision To Language through Graph of Events in Space and Time: An Explainable Self-supervised Approach](https://mlsanity.com/paper/2507.04815/) (2025)
- [Visual Semantic Parsing: From Images to Abstract Meaning Representation](https://mlsanity.com/paper/2210.14862/) (2022)
- [ReCon1M:A Large-scale Benchmark Dataset for Relation Comprehension in Remote Sensing Imagery](https://mlsanity.com/paper/2406.06028/) (2024)
- [LANDMARK: Language-guided Representation Enhancement Framework for Scene Graph Generation](https://mlsanity.com/paper/2303.01080/) (2023)
- [Biasing Like Human: A Cognitive Bias Framework for Scene Graph Generation](https://mlsanity.com/paper/2203.09160/) (2022)
- [SA-VQA: Structured Alignment of Visual and Semantic Representations for Visual Question Answering](https://mlsanity.com/paper/2201.10654/) (2022)

## Abstract (arXiv)

Visual question answering (Visual QA) has attracted significant attention these years. While a variety of algorithms have been proposed, most of them are built upon different combinations of image and language features as well as multi-modal attention and fusion. In this paper, we investigate an alternative approach inspired by conventional QA systems that operate on knowledge graphs. Specifically, we investigate the use of scene graphs derived from images for Visual QA: an image is abstractly represented by a graph with nodes corresponding to object entities and edges to object relationships. We adapt the recently proposed graph network (GN) to encode the scene graph and perform structured reasoning according to the input question. Our empirical studies demonstrate that scene graphs can already capture essential information of images and graph networks have the potential to outperform state-of-the-art Visual QA algorithms but with a much cleaner architecture. By analyzing the features generated by GNs we can further interpret the reasoning process, suggesting a promising direction towards explainable Visual QA.