mlsanityData to 7 Oct 2026

Learning Conditioned Graph Structures for Interpretable Visual Question Answering

arXiv 1806.07243 ↗First posted 19 Jun 2018cs.CVcited by 45

Not on the map. Why?

Abstract (arXiv)

Visual Question answering is a challenging problem requiring a combination of concepts from Computer Vision and Natural Language Processing. Most existing approaches use a two streams strategy, computing image and question features that are consequently merged using a variety of techniques. Nonetheless, very few rely on higher level image representations, which can capture semantic and spatial relationships. In this paper, we propose a novel graph-based approach for Visual Question Answering. Our method combines a graph learner module, which learns a question specific graph representation of the input image, with the recent concept of graph convolutions, aiming to learn image representations that capture question specific interactions. We test our approach on the VQA v2 dataset using a simple baseline architecture enhanced by the proposed graph learner module. We obtain promising results with 66.18% accuracy and demonstrate the interpretability of the proposed method. Code can be found at github.com/aimbrain/vqa-project.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2020: 1; 2021: 12; 2022: 16; 2023: 7; 2024: 5; 2025: 4.

45 papers.

Sort:

By area:

  1. 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 188
  2. 2020Retrieval-Augmented Generation for Code Summarization via Hybrid GNNcited by 89
  3. 2021Pano-AVQA: Grounded Audio-Visual Question Answering on 360^circ Videoscited by 86
  4. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  5. 2021Video as Conditional Graph Hierarchy for Multi-Granular Question Answeringcited by 68
  6. 2022A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspectivecited by 56
  7. 2022Interaction Transformer for Human Reaction Generationcited by 42
  8. 2023Expert Knowledge-Aware Image Difference Graph Representation Learning for Difference-Aware Medical Visual Question Answeringcited by 41
  9. 2022MuKEA: Multimodal Knowledge Extraction and Accumulation for Knowledge-based Visual Question Answeringcited by 36
  10. 2021A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Conceptscited by 33
  11. 2021DialoGraph: Incorporating Interpretable Strategy-Graph Networks into Negotiation Dialoguescited by 25
  12. 2022VQA-GNN: Reasoning with Multimodal Knowledge via Graph Neural Networks for Visual Question Answeringcited by 22
33 more
  1. 2023Unbiased Scene Graph Generation via Two-stage Causal Modelingcited by 21
  2. 2022Visual Perturbation-aware Collaborative Learning for Overcoming the Language Prior Problemcited by 18
  3. 2025A Comprehensive Survey on Self-Interpretable Neural Networkscited by 18
  4. 2021Structured Reordering for Modeling Latent Alignments in Sequence Transductioncited by 17
  5. 2023Interpretable Medical Image Visual Question Answering via Multi-Modal Relationship Graph Learningcited by 16
  6. 2021Fusion-GCN: Multimodal Action Recognition using Graph Convolutional Networkscited by 15
  7. 2021GraphSearchNet: Enhancing GNNs via Capturing Global Dependencies for Semantic Code Searchcited by 15
  8. 2024RepoHyper: Search-Expand-Refine on Semantic Graphs for Repository-Level Code Completioncited by 15
  9. 2022VQA and Visual Reasoning: An Overview of Recent Datasets, Methods and Challengescited by 14
  10. 2025A Causal Adjustment Module for Debiasing Scene Graph Generationcited by 14
  11. 2021Exploiting Relationship for Complex-scene Image Generationcited by 9
  12. 2023Latent Graph Inference with Limited Supervisioncited by 9
  13. 2024A Comprehensive Survey on Visual Question Answering Datasets and Algorithmscited by 8
  14. 2022Dynamic Key-value Memory Enhanced Multi-step Graph Reasoning for Knowledge-based Visual Question Answeringcited by 7
  15. 2024Beyond the Federation: Topology-aware Federated Learning for Generalization to Unseen Clientscited by 7
  16. 2021Scene Graph Embeddings Using Relative Similarity Supervisioncited by 5
  17. 2022QLEVR: A Diagnostic Dataset for Quantificational Language and Elementary Visual Reasoningcited by 5
  18. 2022Semantic-aware Modular Capsule Routing for Visual Question Answeringcited by 5
  19. 2021LiVLR: A Lightweight Visual-Linguistic Reasoning Framework for Video Question Answeringcited by 4
  20. 2021Bilateral Cross-Modality Graph Matching Attention for Feature Fusion in Visual Question Answeringcited by 4
  21. 2022Graph Neural Networks in Computer Vision -- Architectures, Datasets and Common Approachescited by 4
  22. 2022Deep representation learning: Fundamentals, Perspectives, Applications, and Open Challengescited by 3
  23. 2022Enabling Harmonious Human-Machine Interaction with Visual-Context Augmented Dialogue System: A Reviewcited by 2
  24. 2022Memory-Augmented Graph Neural Networks: A Brain-Inspired Reviewcited by 2
  25. 2023Multimodal Prompt Retrieval for Generative Visual Question Answeringcited by 2
  26. 2023A Survey on Interpretable Cross-modal Reasoningcited by 2
  27. 2021Improved RAMEN: Towards Domain Generalization for Visual Question Answeringcited by 1
  28. 2024Ask Questions with Double Hints: Visual Question Generation with Answer-awareness and Region-referencecited by 1
  29. 2025A Reverse Causal Framework to Mitigate Spurious Correlations for Debiasing Scene Graph Generationcited by 1
  30. 2022Main Product Detection with Graph Networks for Fashioncited by 0
  31. 2022SA-VQA: Structured Alignment of Visual and Semantic Representations for Visual Question Answeringcited by 0
  32. 2023VQA with Cascade of Self- and Co-Attention Blockscited by 0
  33. 2025Theoretical Learning Performance of Graph Neural Networks: The Impact of Jumping Connections and Layer-wise Sparsificationcited by 0