Fix Before Search: Benchmarking Agentic Query Visual Pre-processing in Multimodal Retrieval-augmented Generation
On the map: Retrieval-augmented generation → Multimodal RAG
also in 1 more
Abstract (arXiv)
Multimodal Retrieval-Augmented Generation (MRAG) has emerged as a key paradigm for grounding MLLMs with external knowledge. While query pre-processing (e.g., rewriting) is standard in text-based RAG, existing MRAG pipelines predominantly treat visual inputs as static and immutable, implicitly assuming they are noise-free. However, real-world visual queries are often ``imperfect'' -- suffering from geometric distortions, quality degradation, or semantic ambiguity -- leading to catastrophic retrieval failures. To address this gap, we propose V-QPP-Bench, the first comprehensive benchmark dedicated to Visual Query Pre-processing (V-QPP). We formulate V-QPP as an agentic decision-making task where MLLMs must autonomously diagnose imperfections and deploy perceptual tools to refine queries. Our extensive evaluation across 46,700 imperfect queries and diverse MRAG paradigms reveals three critical insights: (1) Vulnerability -- visual imperfections severely degrade both retrieval recall and end-to-end MRAG performance; (2) Restoration Potential & Bottleneck -- while oracle preprocessing recovers near-perfect performance, off-the-shelf MLLMs struggle with tool selection and parameter prediction without specialized training; and (3) Training Enhancement -- supervised fine-tuning enables compact models to achieve comparable or superior performance to larger proprietary models, demonstrating the benchmark's value for developing robust MRAG systems The code is available at https://github.com/phycholosogy/VQQPBench
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 37 of its 47 references that are in the map. First posted on arXiv 13 Feb 2026.
Builds on
37 of 47 references.
Closest ancestors · 5
Same direction, newest first
- 2025A Survey of Multimodal Retrieval-Augmented Generation26 Mar
- 2025Visual-RAG: Benchmarking Text-to-Image Retrieval Augmented Generation for Visual Knowledge Intensive Queries23 Feb
- 2024MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models10 Oct
- 2024RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model16 Feb
- 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text6 Oct
Other references · 32
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2025Qwen3 Technical Reportcited by 8,195
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2024GPT-4o System Cardcited by 4,467
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
- 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
20 more
- 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
- 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
- 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 559
- 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
- 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
- 2024Nomic Embed: Training a Reproducible Long Context Text Embeddercited by 255
- 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generationcited by 144
- 2021WebQA: Multihop and Multimodal QAcited by 109
- 2024NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samplescited by 94
- 2024MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Modelscited by 84
- 2021Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Modelscited by 74
- 2025Process vs. Outcome Reward: Which is Better for Agentic RAG Reinforcement Learningcited by 68
- 2025DeepRetrieval: Hacking Real Search Engines and Retrievers with Large Language Models via Reinforcement Learningcited by 62
- 2023Can Language Models be Instructed to Protect Personal Information?cited by 46
- 2024Insight Over Sight: Exploring the Vision-Knowledge Conflicts in Multimodal LLMscited by 37
- 2025MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughlycited by 29
- 2025AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforced Self-Playcited by 10
- 2025Beyond Text: Unveiling Privacy Vulnerabilities in Multi-modal Retrieval-Augmented Generationcited by 5
- 2025MIRA: A Novel Framework for Fusing Modalities in Medical RAGcited by 4
- 2025CrossCheck-Bench: Diagnosing Compositional Failures in Multimodal Conflict Resolutioncited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Reason Before You Retrieve: Agentic Planning for Multi-modal RAG
- 2025MRAG-Suite: A Diagnostic Evaluation Platform for Visual Retrieval-Augmented Generation
- 2025mRAG: Elucidating the Design Space of Multi-modal Retrieval-Augmented Generation
- 2025CogPlanner: Unveiling the Potential of Agentic Multimodal Retrieval Augmented Generation with Planning
- 2025Efficient Agent: Optimizing Planning Capability for Multimodal Retrieval Augmented Generation
- 2026Efficient Multimodal Planning Agent for Visual Question-Answering
- 2026ATP-Bench: Towards Agentic Tool Planning for MLLM Interleaved Generation
- 2026VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning