mlsanityData to 7 Oct 2026

Adventurer's Treasure Hunt: A Transparent System for Visually Grounded Compositional Visual Question Answering based on Scene Graphs

arXiv 2106.14476 ↗First posted 28 Jun 2021cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

With the expressed goal of improving system transparency and visual grounding in the reasoning process in VQA, we present a modular system for the task of compositional VQA based on scene graphs. Our system is called "Adventurer's Treasure Hunt" (or ATH), named after an analogy we draw between our model's search procedure for an answer and an adventurer's search for treasure. We developed ATH with three characteristic features in mind: 1. By design, ATH allows us to explicitly quantify the impact of each of the sub-components on overall VQA performance, as well as their performance on their individual sub-task. 2. By modeling the search task after a treasure hunt, ATH inherently produces an explicit, visually grounded inference path for the processed question. 3. ATH is the first GQA-trained VQA system that dynamically extracts answers by querying the visual knowledge base directly, instead of selecting one from a specially learned classifier's output distribution over a pre-fixed answer vocabulary. We report detailed results on all components and their contributions to overall VQA performance on the GQA dataset and show that ATH achieves the highest visual grounding score among all examined systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 39 references that are in the map. First posted on arXiv 28 Jun 2021.

Builds on

31 of 39 references.

References · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  6. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  7. 2015VQA: Visual Question Answeringcited by 2,489
  8. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  9. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  10. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  11. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  12. 2015Neural Module Networkscited by 375
19 more
  1. 2017Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answeringcited by 337
  2. 2018Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understandingcited by 304
  3. 201912-in-1: Multi-Task Vision and Language Representation Learningcited by 278
  4. 2015Stacked Attention Networks for Image Question Answeringcited by 273
  5. 2016Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Groundingcited by 243
  6. 2018Compositional Attention Networks for Machine Reasoningcited by 183
  7. 2017Inferring and Executing Programs for Visual Reasoningcited by 130
  8. 2017Learning to Reason: End-to-End Module Networks for Visual Question Answeringcited by 125
  9. 2019Relation-Aware Graph Attention Network for Visual Question Answeringcited by 119
  10. 2019Learning by Abstraction: The Neural State Machinecited by 113
  11. 2018Explainable and Explicit Visual Reasoning over Scene Graphscited by 103
  12. 2017Tips and Tricks for Visual Question Answering: Learnings from the 2017 Challengecited by 76
  13. 2020MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answeringcited by 63
  14. 2019Language-Conditioned Graph Networks for Relational Reasoningcited by 55
  15. 2018Explainable Neural Computation via Stack Neural Module Networkscited by 54
  16. 2018Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoningcited by 51
  17. 2017VQS: Linking Segmentations to Questions and Answers for Supervised Attention in VQA and Question-Focused Semantic Segmentationcited by 50
  18. 2021Found a Reason for me? Weakly-supervised Grounded Visual Question Answering using Capsulescited by 22
  19. 2021GraghVQA: Language-Guided Graph Neural Networks for Graph-based Visual Question Answeringcited by 16

Built on by

Not cited yet.