# Scene Graph Reasoning for Visual Question Answering

arXiv: https://arxiv.org/abs/2007.01072 · HTML: https://mlsanity.com/paper/2007.01072/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2020.

## Built on by (most cited first, 30 of 50)

- [SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities](https://mlsanity.com/paper/2401.12168/) (2024)
- [Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2104.00308/) (2021)
- [Panoptic Scene Graph Generation](https://mlsanity.com/paper/2207.11247/) (2022)
- [SGTR: End-to-end Scene Graph Generation with Transformer](https://mlsanity.com/paper/2112.12970/) (2021)
- [Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2203.09811/) (2022)
- [Relationformer: A Unified Framework for Image-to-Graph Generation](https://mlsanity.com/paper/2203.10202/) (2022)
- [RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulation](https://mlsanity.com/paper/2402.15487/) (2024)
- [From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Models](https://mlsanity.com/paper/2404.00906/) (2024)
- [OODformer: Out-Of-Distribution Detection Transformer](https://mlsanity.com/paper/2107.08976/) (2021)
- [Fine-Grained Predicates Learning for Scene Graph Generation](https://mlsanity.com/paper/2204.02597/) (2022)
- [Iterative Scene Graph Generation with Generative Transformers](https://mlsanity.com/paper/2211.16636/) (2022)
- [InSpire: Vision-Language-Action Models with Intrinsic Spatial Reasoning](https://mlsanity.com/paper/2505.13888/) (2025)
- [HiLo: Exploiting High Low Frequency Relations for Unbiased Panoptic Scene Graph Generation](https://mlsanity.com/paper/2303.15994/) (2023)
- [Look Before You Decide: Prompting Active Deduction of MLLMs for Assumptive Reasoning](https://mlsanity.com/paper/2404.12966/) (2024)
- [OpenPSG: Open-set Panoptic Scene Graph Generation via Large Multimodal Models](https://mlsanity.com/paper/2407.11213/) (2024)
- [SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards](https://mlsanity.com/paper/2511.07403/) (2025)
- [Adaptive Fine-Grained Predicates Learning for Scene Graph Generation](https://mlsanity.com/paper/2207.04602/) (2022)
- [VQA and Visual Reasoning: An Overview of Recent Datasets, Methods and Challenges](https://mlsanity.com/paper/2212.13296/) (2022)
- [Joint learning of object graph and relation graph for visual question answering](https://mlsanity.com/paper/2205.04188/) (2022)
- [VLPrompt: Vision-Language Prompting for Panoptic Scene Graph Generation](https://mlsanity.com/paper/2311.16492/) (2023)
- [Groupwise Query Specialization and Quality-Aware Multi-Assignment for Transformer-based Visual Relationship Detection](https://mlsanity.com/paper/2403.17709/) (2024)
- [Towards Complex-query Referring Image Segmentation: A Novel Benchmark](https://mlsanity.com/paper/2309.17205/) (2023)
- [AerialVG: A Challenging Benchmark for Aerial Visual Grounding by Exploring Positional Relations](https://mlsanity.com/paper/2504.07836/) (2025)
- [LLM Meets Scene Graph: Can Large Language Models Understand and Generate Scene Graphs? A Benchmark and Empirical Study](https://mlsanity.com/paper/2505.19510/) (2025)
- [Learning through structure: towards deep neuromorphic knowledge graph embeddings](https://mlsanity.com/paper/2109.10376/) (2021)
- [Open World Scene Graph Generation using Vision Language Models](https://mlsanity.com/paper/2506.08189/) (2025)
- [Incremental Object Grounding Using Scene Graphs](https://mlsanity.com/paper/2201.01901/) (2022)
- [ROOT: VLM based System for Indoor Scene Understanding and Beyond](https://mlsanity.com/paper/2411.15714/) (2024)
- [RA-SGG: Retrieval-Augmented Scene Graph Generation Framework via Multi-Prototype Learning](https://mlsanity.com/paper/2412.12788/) (2024)
- [ESCA: Contextualizing Embodied Agents via Scene-Graph Generation](https://mlsanity.com/paper/2510.15963/) (2025)

## Abstract (arXiv)

Visual question answering is concerned with answering free-form questions about an image. Since it requires a deep linguistic understanding of the question and the ability to associate it with various objects that are present in the image, it is an ambitious task and requires techniques from both computer vision and natural language processing. We propose a novel method that approaches the task by performing context-driven, sequential reasoning based on the objects and their semantic and spatial relationships present in the scene. As a first step, we derive a scene graph which describes the objects in the image, as well as their attributes and their mutual relationships. A reinforcement agent then learns to autonomously navigate over the extracted scene graph to generate paths, which are then the basis for deriving answers. We conduct a first experimental study on the challenging GQA dataset with manually curated scene graphs, where our method almost reaches the level of human performance.