# Bilinear Graph Networks for Visual Question Answering

arXiv: https://arxiv.org/abs/1907.09815 · HTML: https://mlsanity.com/paper/1907.09815/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jul 2019.

## Built on by (most cited first, 22 of 22)

- [Retrieval Augmented Visual Question Answering with Outside Knowledge](https://mlsanity.com/paper/2210.03809/) (2022)
- [Fine-grained Late-interaction Multi-modal Retrieval for Retrieval Augmented Visual Question Answering](https://mlsanity.com/paper/2309.17133/) (2023)
- [Graph Transformers: A Survey](https://mlsanity.com/paper/2407.09777/) (2024)
- [VQA and Visual Reasoning: An Overview of Recent Datasets, Methods and Challenges](https://mlsanity.com/paper/2212.13296/) (2022)
- [Another Vertical View: A Hierarchical Network for Heterogeneous Trajectory Prediction via Spectrums](https://mlsanity.com/paper/2304.05106/) (2023)
- [A Thousand Words Are Worth More Than a Picture: Natural Language-Centric Outside-Knowledge Visual Question Answering](https://mlsanity.com/paper/2201.05299/) (2022)
- [Motif Graph Neural Network](https://mlsanity.com/paper/2112.14900/) (2021)
- [Answer Mining from a Pool of Images: Towards Retrieval-Based Visual Question Answering](https://mlsanity.com/paper/2306.16713/) (2023)
- [Combo of Thinking and Observing for Outside-Knowledge VQA](https://mlsanity.com/paper/2305.06407/) (2023)
- [A Comprehensive Survey on Visual Question Answering Datasets and Algorithms](https://mlsanity.com/paper/2411.11150/) (2024)
- [Achieving Human Parity on Visual Question Answering](https://mlsanity.com/paper/2111.08896/) (2021)
- [Anchors Aweigh! Sail for Optimal Unified Multi-Modal Representations](https://mlsanity.com/paper/2410.02086/) (2024)
- [Learning Compositional Representation for Few-shot Visual Question Answering](https://mlsanity.com/paper/2102.10575/) (2021)
- [Privacy Preserving Visual Question Answering](https://mlsanity.com/paper/2202.07712/) (2022)
- [Vision-and-Language Pretraining](https://mlsanity.com/paper/2207.01772/) (2022)
- [Multimodal Prompt Retrieval for Generative Visual Question Answering](https://mlsanity.com/paper/2306.17675/) (2023)
- [Answering from Sure to Uncertain: Uncertainty-Aware Curriculum Learning for Video Question Answering](https://mlsanity.com/paper/2401.01510/) (2024)
- [RandAlign: A Parameter-Free Method for Regularizing Graph Convolutional Networks](https://mlsanity.com/paper/2404.09774/) (2024)
- [Syntax Tree Constrained Graph Network for Visual Question Answering](https://mlsanity.com/paper/2309.09179/) (2023)
- [Superpixel Semantics Representation and Pre-training for Vision-Language Task](https://mlsanity.com/paper/2310.13447/) (2023)
- [Admitting Ignorance Helps the Video Question Answering Models to Answer](https://mlsanity.com/paper/2501.08771/) (2025)
- [Improving vision-language alignment with graph spiking hybrid Networks](https://mlsanity.com/paper/2501.19069/) (2025)

## Abstract (arXiv)

This paper revisits the bilinear attention networks in the visual question answering task from a graph perspective. The classical bilinear attention networks build a bilinear attention map to extract the joint representation of words in the question and objects in the image but lack fully exploring the relationship between words for complex reasoning. In contrast, we develop bilinear graph networks to model the context of the joint embeddings of words and objects. Two kinds of graphs are investigated, namely image-graph and question-graph. The image-graph transfers features of the detected objects to their related query words, enabling the output nodes to have both semantic and factual information. The question-graph exchanges information between these output nodes from image-graph to amplify the implicit yet important relationship between objects. These two kinds of graphs cooperate with each other, and thus our resulting model can model the relationship and dependency between objects, which leads to the realization of multi-step reasoning. Experimental results on the VQA v2.0 validation dataset demonstrate the ability of our method to handle the complex questions. On the test-std set, our best single model achieves state-of-the-art performance, boosting the overall accuracy to 72.41%.