mlsanityData to 7 Oct 2026

Understanding the Role of Scene Graphs in Visual Question Answering

arXiv 2101.05479 ↗First posted 14 Jan 2021cs.CV cs.LGcited by 20

Not on the map. Why?

Abstract (arXiv)

Visual Question Answering (VQA) is of tremendous interest to the research community with important applications such as aiding visually impaired users and image-based search. In this work, we explore the use of scene graphs for solving the VQA task. We conduct experiments on the GQA dataset which presents a challenging set of questions requiring counting, compositionality and advanced reasoning capability, and provides scene graphs for a large number of images. We adopt image + question architectures for use with scene graphs, evaluate various scene graph generation techniques for unseen images, propose a training curriculum to leverage human-annotated and auto-generated scene graphs, and build late fusion architectures to learn from multiple image representations. We present a multi-faceted study into the use of scene graphs for VQA, making this work the first of its kind.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 21 of its 28 references that are in the map. First posted on arXiv 14 Jan 2021.

Builds on

21 of 28 references.

References · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2017Mask R-CNNcited by 7,259
  4. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  5. 2015VQA: Visual Question Answeringcited by 2,489
  6. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,571
  7. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  8. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  9. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  10. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  11. 2017Scene Graph Generation by Iterative Message Passingcited by 462
  12. 2017Neural Motifs: Scene Graph Parsing with Global Contextcited by 413
9 more
  1. 2015Neural Module Networkscited by 375
  2. 2020Unbiased Scene Graph Generation from Biased Trainingcited by 355
  3. 2017Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answeringcited by 337
  4. 2018Image Generation from Scene Graphscited by 324
  5. 2018Learning to Compose Dynamic Tree Structures for Visual Contextscited by 258
  6. 2018Compositional Attention Networks for Machine Reasoningcited by 183
  7. 2019Learning by Abstraction: The Neural State Machinecited by 113
  8. 2019Neural Design Network: Graphic Layout Generation with Constraintscited by 83
  9. 2019Controllable and Progressive Image Extrapolationcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 9; 2023: 2; 2024: 4; 2025: 1; 2026: 3.

20 papers.

Sort:

By area:

  1. 2022RU-Net: Regularized Unrolling Network for Scene Graph Generationcited by 28
  2. 2023Learning Situation Hyper-Graphs for Video Question Answeringcited by 15
  3. 2022Joint learning of object graph and relation graph for visual question answeringcited by 13
  4. 2022SSGVS: Semantic Scene Graph-to-Video Synthesiscited by 8
  5. 2024VeriGraph: Scene Graphs for Execution Verifiable Robot Planningcited by 7
  6. 2022Visual Semantic Parsing: From Images to Abstract Meaning Representationcited by 4
  7. 2023Fine-Grained is Too Coarse: A Novel Data-Centric Approach for Efficient Scene Graph Generationcited by 4
  8. 2024Joint Generative Modeling of Grounded Scene Graphs and Images via Diffusion Modelscited by 4
  9. 2022Symbolic image detection using scene and knowledge graphscited by 3
  10. 2024REACT: Real-time Efficiency and Accuracy Compromise for Tradeoffs in Scene Graph Generationcited by 3
  11. 2021UNITER-Based Situated Coreference Resolution with Rich Multimodal Inputcited by 2
  12. 2022SGRAM: Improving Scene Graph Parsing via Abstract Meaning Representationcited by 2
8 more
  1. 2026Bridging Structure and Language: Graph-Based Visual Reasoning for Autonomous Road Understandingcited by 2
  2. 2022Grounding Scene Graphs on Natural Images via Visio-Lingual Message Passingcited by 1
  3. 2025CoSPlan: Corrective Sequential Planning via Scene Graph Incremental Updatescited by 1
  4. 2026REACT++: Efficient Cross-Attention for Real-Time Scene Graph Generationcited by 1
  5. 2022SA-VQA: Structured Alignment of Visual and Semantic Representations for Visual Question Answeringcited by 0
  6. 2022SrTR: Self-reasoning Transformer with Visual-linguistic Knowledge for Scene Graph Generationcited by 0
  7. 2024Discrete Subgraph Sampling for Interpretable Graph based Visual Question Answeringcited by 0
  8. 2026DSFlash: Comprehensive Panoptic Scene Graph Generation in Realtimecited by 0