# GraghVQA: Language-Guided Graph Neural Networks for Graph-based Visual Question Answering

arXiv: https://arxiv.org/abs/2104.10283 · HTML: https://mlsanity.com/paper/2104.10283/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 24 references that are in the map.
- First posted on arXiv 20 Apr 2021.

## Builds on (most cited first, 20 of 20)

- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [SPICE: Semantic Propositional Image Caption Evaluation](https://mlsanity.com/paper/1607.08822/) (2016)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [Image Generation from Scene Graphs](https://mlsanity.com/paper/1804.01622/) (2018)
- [Analyzing the Behavior of Visual Question Answering Models](https://mlsanity.com/paper/1606.07356/) (2016)
- [Relation-Aware Graph Attention Network for Visual Question Answering](https://mlsanity.com/paper/1903.12314/) (2019)
- [Learning by Abstraction: The Neural State Machine](https://mlsanity.com/paper/1907.03950/) (2019)
- [Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?](https://mlsanity.com/paper/1606.03556/) (2016)
- [Did the Model Understand the Question?](https://mlsanity.com/paper/1805.05492/) (2018)
- [Language-Conditioned Graph Networks for Relational Reasoning](https://mlsanity.com/paper/1905.04405/) (2019)
- [ALICE: Active Learning with Contrastive Natural Language Explanations](https://mlsanity.com/paper/2009.10259/) (2020)
- [Neuro-Symbolic Visual Reasoning: Disentangling "Visual" from "Reasoning"](https://mlsanity.com/paper/2006.11524/) (2020)
- [Situation Recognition with Graph Neural Networks](https://mlsanity.com/paper/1708.04320/) (2017)
- [Beyond User Self-Reported Likert Scale Ratings: A Comparison Model for Automatic Dialog Evaluation](https://mlsanity.com/paper/2005.10716/) (2020)
- [MOSS: End-to-End Dialog System Framework with Modular Supervision](https://mlsanity.com/paper/1909.05528/) (2019)
- [LRTA: A Transparent Neural-Symbolic Reasoning Framework with Modular Supervision for Visual Question Answering](https://mlsanity.com/paper/2011.10731/) (2020)
- [Neural Group Testing to Accelerate Deep Learning](https://mlsanity.com/paper/2011.10704/) (2020)

## Built on by (most cited first, 16 of 16)

- [Towards Semantic Equivalence of Tokenization in Multimodal LLM](https://mlsanity.com/paper/2406.05127/) (2024)
- [VQA-GNN: Reasoning with Multimodal Knowledge via Graph Neural Networks for Visual Question Answering](https://mlsanity.com/paper/2205.11501/) (2022)
- [Multimodal Graph Transformer for Multimodal Question Answering](https://mlsanity.com/paper/2305.00581/) (2023)
- [HERALD: An Annotation Efficient Method to Detect User Disengagement in Social Conversations](https://mlsanity.com/paper/2106.00162/) (2021)
- [DSPNet: Dual-vision Scene Perception for Robust 3D Question Answering](https://mlsanity.com/paper/2503.03190/) (2025)
- [Symbolic image detection using scene and knowledge graphs](https://mlsanity.com/paper/2206.04863/) (2022)
- [Visually Grounded VQA by Lattice-based Retrieval](https://mlsanity.com/paper/2211.08086/) (2022)
- [REACT: Real-time Efficiency and Accuracy Compromise for Tradeoffs in Scene Graph Generation](https://mlsanity.com/paper/2405.16116/) (2024)
- [Goal-Oriented Semantic Communication for Wireless Visual Question Answering](https://mlsanity.com/paper/2411.02452/) (2024)
- [SelfGraphVQA: A Self-Supervised Graph Neural Network for Scene-based Question Answering](https://mlsanity.com/paper/2310.01842/) (2023)
- [Intrinsic Subgraph Generation for Interpretable Graph based Visual Question Answering](https://mlsanity.com/paper/2403.17647/) (2024)
- [Bridging the Gap Between Multimodal Foundation Models and World Models](https://mlsanity.com/paper/2510.03727/) (2025)
- [REACT++: Efficient Cross-Attention for Real-Time Scene Graph Generation](https://mlsanity.com/paper/2603.06386/) (2026)
- [Adventurer's Treasure Hunt: A Transparent System for Visually Grounded Compositional Visual Question Answering based on Scene Graphs](https://mlsanity.com/paper/2106.14476/) (2021)
- [Unified Lattice Graph Fusion for Chinese Named Entity Recognition](https://mlsanity.com/paper/2312.16917/) (2023)
- [DSFlash: Comprehensive Panoptic Scene Graph Generation in Realtime](https://mlsanity.com/paper/2603.10538/) (2026)

## Abstract (arXiv)

Images are more than a collection of objects or attributes -- they represent a web of relationships among interconnected objects. Scene Graph has emerged as a new modality for a structured graphical representation of images. Scene Graph encodes objects as nodes connected via pairwise relations as edges. To support question answering on scene graphs, we propose GraphVQA, a language-guided graph neural network framework that translates and executes a natural language question as multiple iterations of message passing among graph nodes. We explore the design space of GraphVQA framework, and discuss the trade-off of different design choices. Our experiments on GQA dataset show that GraphVQA outperforms the state-of-the-art model by a large margin (88.43% vs. 94.78%).