mlsanityData to 7 Oct 2026

Coarse-to-Fine Reasoning for Visual Question Answering

arXiv 2110.02526 ↗First posted 6 Oct 2021cs.CVcited by 25

Not on the map. Why?

Abstract (arXiv)

Bridging the semantic gap between image and question is an important step to improve the accuracy of the Visual Question Answering (VQA) task. However, most of the existing VQA methods focus on attention mechanisms or visual relations for reasoning the answer, while the features at different semantic levels are not fully utilized. In this paper, we present a new reasoning framework to fill the gap between visual features and semantic clues in the VQA task. Our method first extracts the features and predicates from the image and question. We then propose a new reasoning framework to effectively jointly learn these features and predicates in a coarse-to-fine manner. The intensively experimental results on three large-scale VQA datasets show that our proposed approach achieves superior accuracy comparing with other state-of-the-art methods. Furthermore, our reasoning framework also provides an explainable way to understand the decision of the deep neural network when predicting the answer.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 54 of its 70 references that are in the map. First posted on arXiv 6 Oct 2021.

Builds on

54 of 70 references.

References · 54

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  4. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  5. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  6. 2015VQA: Visual Question Answeringcited by 2,489
  7. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  8. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  9. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  10. 2019The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervisioncited by 369
  11. 2019Deep Modular Co-Attention Networks for Visual Question Answeringcited by 338
  12. 2015Visual7W: Grounded Question Answering in Imagescited by 315
38 more
  1. 2017Relation Networks for Object Detectioncited by 305
  2. 2018Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understandingcited by 304
  3. 2018Bilinear Attention Networkscited by 277
  4. 2015Stacked Attention Networks for Image Question Answeringcited by 273
  5. 2018Exploring Visual Relationship for Image Captioningcited by 215
  6. 2016FVQA: Fact-based Visual Question Answeringcited by 181
  7. 2019Knowledge-Embedded Routing Network for Scene Graph Generationcited by 163
  8. 2020In Defense of Grid Features for Visual Question Answeringcited by 155
  9. 2017IQA: Visual Question Answering in Interactive Environmentscited by 154
  10. 2017AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detectioncited by 126
  11. 2016Graph-Structured Representations for Visual Question Answeringcited by 123
  12. 2019Relation-Aware Graph Attention Network for Visual Question Answeringcited by 119
  13. 2018Dynamic Fusion with Intra- and Inter- Modality Attention Flow for Visual Question Answeringcited by 113
  14. 2019Learning by Abstraction: The Neural State Machinecited by 113
  15. 2018Neighbourhood Watch: Referring Expression Comprehension via Language-guided Graph Attention Networkscited by 102
  16. 2019Scene Graph Generation with External Knowledge and Image Reconstructioncited by 102
  17. 2017Tips and Tricks for Visual Question Answering: Learnings from the 2017 Challengecited by 76
  18. 2021Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Modelscited by 74
  19. 2019MUREL: Multimodal Relational Reasoning for Visual Question Answeringcited by 68
  20. 2018Pythia v0.1: the Winning Entry to the VQA Challenge 2018cited by 62
  21. 2019Self-Critical Reasoning for Robust Visual Question Answeringcited by 61
  22. 2021Multiple Meta-model Quantifying for Medical Visual Question Answeringcited by 54
  23. 2018Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoningcited by 51
  24. 2020Multi-Modal Graph Neural Network for Joint Reasoning on Vision and Scene Textcited by 46
  25. 2019Meta Module Network for Compositional Visual Reasoningcited by 39
  26. 2020Unshuffling Data for Improved Generalizationcited by 35
  27. 2020Neuro-Symbolic Visual Reasoning: Disentangling "Visual" from "Reasoning"cited by 35
  28. 2017Deep Variation-structured Reinforcement Learning for Visual Relationship and Attribute Detectioncited by 28
  29. 2019Compact Trilinear Interaction for Visual Question Answeringcited by 24
  30. 2018R-VQA: Learning Visual Relation Facts with Semantic Attention for Visual Question Answeringcited by 23
  31. 2021Found a Reason for me? Weakly-supervised Grounded Visual Question Answering using Capsulescited by 22
  32. 2020Autonomous Navigation in Complex Environments with Deep Multimodal Fusion Networkcited by 16
  33. 2020Transformation Driven Visual Reasoningcited by 14
  34. 2021Graph-based Person Signature for Person Re-Identificationscited by 14
  35. 2021Light-weight Deformable Registration using Adversarial Learning with Distilling Knowledgecited by 12
  36. 2018Learning Answer Embeddings for Visual Question Answeringcited by 11
  37. 2020Cross-Modality Relevance for Reasoning on Language and Visioncited by 10
  38. 2021'Just because you are right, doesn't mean I am wrong': Overcoming a Bottleneck in the Development and Evaluation of Open-Ended Visual Question Answering (VQA) Taskscited by 10

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 3; 2023: 10; 2024: 5; 2025: 5.

25 papers.

Sort:

By area:

  1. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
  2. 2023LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Modelscited by 217
  3. 2021MAGMA -- Multimodal Augmentation of Generative Models through Adapter-based Finetuningcited by 85
  4. 2021Deep Federated Learning for Autonomous Drivingcited by 54
  5. 2023Robust Visual Question Answering: Datasets, Methods, and Future Challengescited by 23
  6. 2022VQA-GNN: Reasoning with Multimodal Knowledge via Graph Neural Networks for Visual Question Answeringcited by 22
  7. 2023Visual Question Answering: A Survey on Techniques and Common Trends in Recent Literaturecited by 20
  8. 2024SearchLVLMs: A Plug-and-Play Framework for Augmenting Large Vision-Language Models by Searching Up-to-Date Internet Knowledgecited by 20
  9. 2024Enhancing Visual Question Answering through Ranking-Based Hybrid Training and Multimodal Fusioncited by 16
  10. 2024Towards Truly Zero-shot Compositional Visual Reasoning with LLMs as Programmerscited by 14
  11. 2023Towards Top-Down Reasoning: An Explainable Multi-Agent Approach for Visual Question Answeringcited by 11
  12. 2022Fine-Grained Visual Classification using Self Assessment Classifiercited by 8
13 more
  1. 2024FlexCap: Describe Anything in Images in Controllable Detailcited by 7
  2. 2024SADL: An Effective In-Context Learning Method for Compositional Visual QAcited by 5
  3. 2023A Survey on Image-text Multimodal Modelscited by 4
  4. 2022Towards Reasoning-Aware Explainable VQAcited by 2
  5. 2023A Survey on Interpretable Cross-modal Reasoningcited by 2
  6. 2023SelfGraphVQA: A Self-Supervised Graph Neural Network for Scene-based Question Answeringcited by 2
  7. 2023A Stepwise Distillation Learning Strategy for Non-differentiable Visual Programming Frameworks on Visual Reasoning Taskscited by 1
  8. 2025Predicate Hierarchies Improve Few-Shot State Classificationcited by 1
  9. 2025Efficiently Enhancing General Agents With Hierarchical-categorical Memorycited by 1
  10. 2025Physics-Based Benchmarking Metrics for Multimodal Synthetic Imagescited by 1
  11. 2023Joint Adaptive Representations for Image-Language Learningcited by 0
  12. 2025Multi-Sourced Compositional Generalization in Visual Question Answeringcited by 0
  13. 2025A Framework for Generating Artificial Datasets to Validate Absolute and Relative Position Conceptscited by 0