mlsanityData to 14 Oct 2026

Interpretable Visual Question Answering by Visual Grounding from Attention Supervision Mining

arXiv 1808.00265 ↗First posted 1 Aug 2018cs.CV cs.AI cs.CLcited by 28

Not on the map. Why?

Abstract (arXiv)

A key aspect of VQA models that are interpretable is their ability to ground their answers to relevant regions in the image. Current approaches with this capability rely on supervised learning and human annotated groundings to train attention mechanisms inside the VQA architecture. Unfortunately, obtaining human annotations specific for visual grounding is difficult and expensive. In this work, we demonstrate that we can effectively train a VQA architecture with grounding supervision that can be automatically obtained from available region descriptions and object annotations. We also show that our model trained with this mined supervision generates visual groundings that achieve a higher correlation with respect to manually-annotated groundings, meanwhile achieving state-of-the-art VQA accuracy.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). First posted on arXiv 1 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 5; 2022: 9; 2023: 9; 2024: 2; 2025: 3.

28 papers.

Sort:

By area:

  1. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 603
  2. 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 51
  3. 2022Grounding Answers for Visual Questions Asked by Visually Impaired Peoplecited by 47
  4. 2022Going Beyond XAI: A Systematic Survey for Explanation-Guided Learningcited by 36
  5. 2021Greedy Gradient Ensemble for Robust Visual Question Answeringcited by 34
  6. 2023Multimodal Explainable Artificial Intelligence: A Comprehensive Review of Methodological Advances and Future Research Directionscited by 25
  7. 2021Counterfactual Samples Synthesizing and Training for Robust Visual Question Answeringcited by 22
  8. 2022RES: A Robust Framework for Guiding Visual Explanationcited by 22
  9. 2024Surgical-VQLA++: Adversarial Contrastive Learning for Calibrated Robust Visual Question-Localized Answering in Robotic Surgerycited by 17
  10. 2023VQA Therapy: Exploring Answer Differences by Visually Grounding Answerscited by 14
  11. 2022A Thousand Words Are Worth More Than a Picture: Natural Language-Centric Outside-Knowledge Visual Question Answeringcited by 13
  12. 2021Answer Questions with Right Image Regions: A Visual Attention Regularization Approachcited by 9
16 more
  1. 2022Weakly Supervised Grounding for VQA in Vision-Language Transformerscited by 8
  2. 2023Sentence Attention Blocks for Answer Groundingcited by 8
  3. 2024A Comprehensive Survey on Visual Question Answering Datasets and Algorithmscited by 8
  4. 2021More Than Just Attention: Improving Cross-Modal Attentions with Contrastive Constraints for Image-Text Matchingcited by 5
  5. 2022COIN: Counterfactual Image Generation for VQA Interpretationcited by 4
  6. 2023Interpretable Visual Question Answering via Reasoning Supervisioncited by 4
  7. 2022CNN Attention Guidance for Improved Orthopedics Radiographic Fracture Classificationcited by 2
  8. 2023Learning to Agree on Vision Attention for Visual Commonsense Reasoningcited by 2
  9. 2023Interpreting Vision and Language Generative Models with Semantic Visual Priorscited by 2
  10. 2023A Joint Study of Phrase Grounding and Task Performance in Vision and Language Modelscited by 2
  11. 2022An experimental study of the vision-bottleneck in VQAcited by 1
  12. 2022Attention in Reasoning: Dataset, Analysis, and Modelingcited by 1
  13. 2023Continual Referring Expression Comprehension via Dual Modular Memorizationcited by 1
  14. 2025Efficient Text-Attributed Graph Learning through Selective Annotation and Graph Alignmentcited by 0
  15. 2025Integrating Object Interaction Self-Attention and GAN-Based Debiasing for Visual Question Answeringcited by 0
  16. 2025Calibrating Transformer Attention via Task-Space Sensitivity Feedbackcited by 0