mlsanityData to 7 Oct 2026

REVIVE: Regional Visual Representation Matters in Knowledge-Based Visual Question Answering

arXiv 2206.01201 ↗First posted 2 Jun 2022cs.CV cs.CLcited by 58

Not on the map. Why?

Abstract (arXiv)

This paper revisits visual representation in knowledge-based visual question answering (VQA) and demonstrates that using regional information in a better way can significantly improve the performance. While visual representation is extensively studied in traditional VQA, it is under-explored in knowledge-based VQA even though these two tasks share the common spirit, i.e., rely on visual input to answer the question. Specifically, we observe that in most state-of-the-art knowledge-based VQA methods: 1) visual features are extracted either from the whole image or in a sliding window manner for retrieving knowledge, and the important relationship within/among object regions is neglected; 2) visual features are not well utilized in the final answering model, which is counter-intuitive to some extent. Based on these observations, we propose a new knowledge-based VQA method REVIVE, which tries to utilize the explicit information of object regions not only in the knowledge retrieval stage but also in the answering model. The key motivation is that object regions and inherent relationship are important for knowledge-based VQA. We perform extensive experiments on the standard OK-VQA dataset and achieve new state-of-the-art performance, i.e., 58.0% accuracy, surpassing previous state-of-the-art method by a large margin (+3.6%). We also conduct detailed analysis and show the necessity of regional information in different framework components for knowledge-based VQA. Code is publicly available at https://github.com/yzleroy/REVIVE.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). Builds on 35 of its 42 references that are in the map. First posted on arXiv 2 Jun 2022.

Builds on

35 of 42 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  8. 2015VQA: Visual Question Answeringcited by 2,489
  9. 2017Billion-scale similarity search with GPUscited by 2,477
  10. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  11. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  12. 2021Grounded Language-Image Pre-trainingcited by 1,147
23 more
  1. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  2. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  3. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  4. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  5. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  6. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  7. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 645
  8. 2017Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answeringcited by 337
  9. 2021An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQAcited by 298
  10. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  11. 2016FVQA: Fact-based Visual Question Answeringcited by 181
  12. 2021KAT: A Knowledge Augmented Transformer for Vision-and-Languagecited by 125
  13. 2020KRISP: Integrating Implicit and Symbolic Knowledge for Open-Domain Knowledge-Based VQAcited by 124
  14. 2015Explicit Knowledge-based Reasoning for Visual Question Answeringcited by 85
  15. 2021Multi-Modal Answer Validation for Knowledge-Based VQAcited by 78
  16. 2019KnowIT VQA: Answering Knowledge-Based Questions about Videoscited by 53
  17. 2020Mucko: Multi-Layer Cross-Modal Knowledge Reasoning for Fact-based Visual Question Answeringcited by 51
  18. 2021Weakly-Supervised Visual-Retriever-Reader for Knowledge-based Question Answeringcited by 50
  19. 2021UFO: A UniFied TransfOrmer for Vision-Language Representation Learningcited by 45
  20. 2022Gender and Racial Bias in Visual Question Answering Datasetscited by 39
  21. 2022Pseudo-Q: Generating Pseudo Language Queries for Visual Groundingcited by 33
  22. 2018Straight to the Facts: Learning Knowledge Base Retrieval for Factual Visual Question Answeringcited by 30
  23. 2020Knowledge-Based Video Question Answering with Unsupervised Scene Descriptionscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 16; 2024: 18; 2025: 12; 2026: 6.

58 papers.

Sort:

By area:

  1. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
  2. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 203
  3. 2022REVEAL: Retrieval-Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowledge Memorycited by 122
  4. 2022From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Modelscited by 114
  5. 2023Encyclopedic VQA: Visual questions about detailed properties of fine-grained categoriescited by 105
  6. 2022PromptCap: Prompt-Guided Task-Aware Image Captioningcited by 104
  7. 2023X-InstructBLIP: A Framework for aligning X-Modal instruction-aware representations to LLMs and Emergent Cross-modal Reasoningcited by 78
  8. 2024Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agentcited by 65
  9. 2024What Makes Multimodal In-Context Learning Work?cited by 54
  10. 2023Fine-grained Late-interaction Multi-modal Retrieval for Retrieval Augmented Visual Question Answeringcited by 51
  11. 2024Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Surveycited by 50
  12. 2023See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge-based Visual Reasoningcited by 49
38 more
  1. 2024Situational Awareness Matters in 3D Vision Language Reasoningcited by 32
  2. 2023Text-Driven Image Editing via Learnable Regionscited by 31
  3. 2023Prophet: Prompting Large Language Models with Complementary Answer Heuristics for Knowledge-based Visual Question Answeringcited by 21
  4. 2023Improving Zero-shot Visual Question Answering via Large Language Models with Reasoning Question Promptscited by 19
  5. 2022LaKo: Knowledge-driven Visual Question Answering via Late Knowledge-to-Text Injectioncited by 18
  6. 2024Cross-Modal Consistency in Multimodal Large Language Modelscited by 17
  7. 2023Generate then Select: Open-ended Visual Question Answering Guided by World Knowledgecited by 16
  8. 2024VLLMs Provide Better Context for Emotion Understanding Through Common Sense Reasoningcited by 15
  9. 2022SMAUG: Sparse Masked Autoencoder for Efficient Video-Language Pre-trainingcited by 14
  10. 2024DreamPolisher: Towards High-Quality Text-to-3D Generation via Geometric Diffusioncited by 12
  11. 2023Zero-shot Visual Question Answering with Language Model Feedbackcited by 11
  12. 2023Towards Top-Down Reasoning: An Explainable Multi-Agent Approach for Visual Question Answeringcited by 11
  13. 2024What Makes for Good Image Captions?cited by 11
  14. 2023Combo of Thinking and Observing for Outside-Knowledge VQAcited by 9
  15. 2024GeReA: Question-Aware Prompt Captions for Knowledge-based Visual Question Answeringcited by 9
  16. 2023AVIS: Autonomous Visual Information Seeking with Large Language Model Agentcited by 8
  17. 2024Generative AI and Process Systems Engineering: The Next Frontiercited by 8
  18. 2022Improving Commonsense in Vision-Language Models via Knowledge Graph Riddlescited by 7
  19. 2024HAMMR: HierArchical MultiModal React agents for generic VQAcited by 7
  20. 2025FilterRAG: Zero-Shot Informed Retrieval-Augmented Generation to Mitigate Hallucinations in VQAcited by 5
  21. 2025Describe Anything Model for Visual Question Answering on Text-rich Imagescited by 5
  22. 2024Find The Gap: Knowledge Base Reasoning For Visual Question Answeringcited by 4
  23. 2024Olympus: A Universal Task Router for Computer Vision Taskscited by 4
  24. 2023SAIR: Learning Semantic-aware Implicit Representationcited by 3
  25. 2024Q&A Prompts: Discovering Rich Visual Clues through Mining Question-Answer Prompts for VQA requiring Diverse World Knowledgecited by 3
  26. 2024Knowledge Condensation and Reasoning for Knowledge-based VQAcited by 3
  27. 2026Multimodal Adaptive Retrieval Augmented Generation through Internal Representation Learningcited by 3
  28. 2023The Contribution of Knowledge in Visiolinguistic Learning: A Survey on Tasks and Challengescited by 2
  29. 2024Disentangling Knowledge-based and Visual Reasoning by Question Decomposition in KB-VQAcited by 2
  30. 2025MV-CoRe: Multimodal Visual-Conceptual Reasoning for Complex Visual Question Answeringcited by 2
  31. 2025Breaking the Visual Shortcuts in Multimodal Knowledge-Based Visual Question Answeringcited by 2
  32. 2026Pixel-Grounded Retrieval for Knowledgeable Large Multimodal Modelscited by 2
  33. 2025Computed Tomography Visual Question Answering with Cross-modal Feature Graphingcited by 1
  34. 2025SCRA-VQA: Summarized Caption-Rerank for Augmented Large Language Models in Visual Question Answeringcited by 1
  35. 2025StaR-KVQA: Structured Reasoning Traces for Implicit-Knowledge Visual Question Answeringcited by 1
  36. 2025From Hindsight to Foresight: Self-Encouraged Hindsight Distillation for Knowledge-based Visual Question Answeringcited by 1
  37. 2024Towards Multilingual Audio-Visual Question Answeringcited by 0
  38. 2024Generating Event-oriented Attribution for Movies via Two-Stage Prefix-Enhanced Multimodal LLMcited by 0

50 most cited of 58 All 58 (JSON)