mlsanityData to 14 Oct 2026

Progressive Multimodal Search and Reasoning for Knowledge-Intensive Visual Question Answering

arXiv 2509.00798 ↗First posted 31 Aug 2025cs.CV cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Knowledge-intensive visual question answering (VQA) requires external knowledge beyond image content, demanding precise visual grounding and coherent integration of visual and textual information. Although multimodal retrieval-augmented generation has achieved notable advances by incorporating external knowledge bases, existing approaches largely adopt single-pass frameworks that often fail to acquire sufficient knowledge and lack mechanisms to revise misdirected reasoning. We propose PMSR (Progressive Multimodal Search and Reasoning), a framework that progressively constructs a structured reasoning trajectory to enhance both knowledge acquisition and synthesis. PMSR uses dual-scope queries conditioned on both the latest record and the trajectory to retrieve diverse knowledge from heterogeneous knowledge bases. The retrieved evidence is then synthesized into compact records via compositional reasoning. This design facilitates controlled iterative refinement, which supports more stable reasoning trajectories with reduced error propagation. Extensive experiments across six diverse benchmarks (Encyclopedic-VQA, InfoSeek, MMSearch, LiveVQA, FVQA, and OK-VQA) demonstrate that PMSR consistently improves both retrieval recall and end-to-end answer accuracy.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 40 of its 61 references that are in the map. First posted on arXiv 31 Aug 2025.

Builds on

40 of 61 references.

References · 40

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  4. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  5. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  6. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  7. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  8. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,121
  9. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  10. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  11. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 369
  12. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 281
28 more
  1. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 207
  2. 2023UniIR: Training and Benchmarking Universal Multimodal Information Retrieverscited by 166
  3. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Researchcited by 166
  4. 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 161
  5. 2025MMSearch-R1: Incentivizing LMMs to Searchcited by 124
  6. 2023Encyclopedic VQA: Visual questions about detailed properties of fine-grained categoriescited by 107
  7. 2024LamRA: Large Multimodal Model as Your Advanced Retrieval Assistantcited by 106
  8. 2024MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Enginescited by 101
  9. 2025Words or Vision: Do Vision-Language Models Have Blind Faith in Text?cited by 89
  10. 2024Wiki-LLaVA: Hierarchical Retrieval-Augmented Generation for Multimodal LLMscited by 80
  11. 2024RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generationcited by 75
  12. 2024Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agentcited by 65
  13. 2024EchoSight: Advancing Visual-Language Models with Wiki Knowledgecited by 62
  14. 2024Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questionscited by 54
  15. 2021Weakly-Supervised Visual-Retriever-Reader for Knowledge-based Question Answeringcited by 50
  16. 2024MLLM Is a Strong Reranker: Advancing Multimodal Retrieval-augmented Generation via Knowledge-enhanced Reranking and Noise-injected Trainingcited by 50
  17. 2024RA-ISF: Learning to Answer and Understand from Retrieval Augmentation via Iterative Self-Feedbackcited by 46
  18. 2024PreFLMR: Scaling Up Fine-Grained Late-Interaction Multi-modal Retrieverscited by 42
  19. 2025s3: You Don't Need That Much Data to Train a Search Agent via RLcited by 42
  20. 2024Retrieve, Summarize, Plan: Advancing Multi-hop Question Answering with an Iterative Approachcited by 40
  21. 2025Seeking and Updating with Live Visual Knowledgecited by 24
  22. 2023End-to-end Knowledge Retrieval with Multi-modal Queriescited by 23
  23. 2024mR2AG: Multimodal Retrieval-Reflection-Augmented Generation for Knowledge-Based VQAcited by 19
  24. 2025MMKB-RAG: A Multi-Modal Knowledge-Based Retrieval-Augmented Generation Frameworkcited by 18
  25. 2025Retrieval-Augmented Visual Question Answering via Built-in Autoregressive Search Enginescited by 13
  26. 2025Recurrence-Enhanced Vision-and-Language Transformers for Robust Multimodal Document Retrievalcited by 13
  27. 2025LevelRAG: Enhancing Retrieval-Augmented Generation with Multi-hop Logic Planning over Rewriting Augmented Searcherscited by 11
  28. 2025OMGM: Orchestrate Multiple Granularities and Modalities for Efficient Multimodal Retrievalcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026WikiSeeker: Rethinking the Role of Vision-Language Models in Knowledge-Based Visual Question Answeringcited by 1
  2. 2026Caesar: Deep Agentic Web Exploration for Creative Answer Synthesiscited by 1
  3. 2026EviRover: Reinforcing Agentic Perception Beyond a Glancecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ProMSA:Progressive Multimodal Search Agents for Knowledge-Based Visual Question Answering
  2. 2025ReAG: Reasoning-Augmented Generation for Knowledge-based Visual Question Answering
  3. 2026Enhancing Visual Question Answering with Multimodal LLMs via Chain-of-Question Guided Retrieval-Augmented Generation
  4. 2025Knowledge-based Visual Question Answer with Multimodal Processing, Retrieval and Filtering
  5. 2026MaS-VQA: A Mask-and-Select Framework for Knowledge-Based Visual Question Answering
  6. 2026M3-VQA: A Benchmark for Multimodal, Multi-Entity, Multi-Hop Visual Question Answering
  7. 2025Re-ranking Reasoning Context with Tree Search Makes Large Vision-Language Models Stronger
  8. 2026Lost at the End: Primacy Bias in Multimodal Retrieval-Augmented Question Answering