V-Retrver: Evidence-Driven Agentic Reasoning for Universal Multimodal Retrieval
On the map: LLM reasoning → Multimodal reasoning
also in 2 more
Abstract (arXiv)
Multimodal Large Language Models (MLLMs) have recently been applied to universal multimodal retrieval, where Chain-of-Thought (CoT) reasoning improves candidate reranking. However, existing approaches remain largely language-driven, relying on static visual encodings and lacking the ability to actively verify fine-grained visual evidence, which often leads to speculative reasoning in visually ambiguous cases. We propose V-Retrver, an evidence-driven retrieval framework that reformulates multimodal retrieval as an agentic reasoning process grounded in visual inspection. V-Retrver enables an MLLM to selectively acquire visual evidence during reasoning via external visual tools, performing a multimodal interleaved reasoning process that alternates between hypothesis generation and targeted visual verification.To train such an evidence-gathering retrieval agent, we adopt a curriculum-based learning strategy combining supervised reasoning activation, rejection-based refinement, and reinforcement learning with an evidence-aligned objective. Experiments across multiple multimodal retrieval benchmarks demonstrate consistent improvements in retrieval accuracy (with 23.0% improvements on average), perception-driven reasoning reliability, and generalization.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 78 of its 87 references that are in the map. First posted on arXiv 5 Feb 2026.
Builds on
78 of 87 references.
Closest ancestors · 8
Same direction, newest first
- 2025AdaTooler-V: Adaptive Tool-Use for Images and Videos18 Dec
- 2025Knowing the Answer Isn't Enough: Fixing Reasoning Path Failures in LVLMs6 Dec
- 2025Retrv-R1: A Reasoning-Driven MLLM Framework for Universal and Efficient Multimodal Retrieval3 Oct
- 2025Vision-EKIPL: External Knowledge-Infused Policy Learning for Visual Reasoning7 Jun
- 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
- 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
Other references · 70
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2023Qwen Technical Reportcited by 3,273
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
38 more
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
- 2025s1: Simple test-time scalingcited by 1,311
- 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
- 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
- 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677
- 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 545
- 2016Visual Dialogcited by 377
- 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 285
- 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 203
- 2024LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Imagescited by 193
- 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 188
- 2016Visual Storytellingcited by 176
- 2023Zero-Shot Composed Image Retrieval with Textual Inversioncited by 171
- 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 166
- 2023UniIR: Training and Benchmarking Universal Multimodal Information Retrieverscited by 165
- 2025LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Tokencited by 164
- 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 159
- 2023Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrievalcited by 155
- 2021Probabilistic Embeddings for Cross-Modal Retrievalcited by 150
- 2024E5-V: Universal Embeddings with Multimodal Large Language Modelscited by 123
- 2024MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructionscited by 119
- 2023Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entitiescited by 112
- 2022Task-aware Retrieval with Instructionscited by 107
- 2024Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Expertscited by 107
- 2024LamRA: Large Multimodal Model as Your Advanced Retrieval Assistantcited by 106
- 2023Encyclopedic VQA: Visual questions about detailed properties of fine-grained categoriescited by 105
- 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performancecited by 94
- 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Usecited by 92
- 2024MegaPairs: Massive Data Synthesis For Universal Multimodal Retrievalcited by 84
- 2023CLIP for All Things Zero-Shot Sketch-Based Image Retrieval, Fine-Grained or Notcited by 79
- 2025Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMscited by 73
- 2022Universal Vision-Language Dense Retrieval: Learning A Unified Representation Space for Multi-Modal Retrievalcited by 57
- 2025LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learningcited by 54
- 2023Fine-grained Late-interaction Multi-modal Retrieval for Retrieval Augmented Visual Question Answeringcited by 51
50 most cited of 70 All 70 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026VimRAG: Navigating Massive Visual Context in Retrieval-Augmented Generation via Multimodal Memory Graph
- 2026Visual-Seeker: Towards Visual-Native Multimodal Agentic Search via Active Visual Reasoning
- 2026Reasoning-Augmented Representations for Multimodal Retrieval
- 2026MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory
- 2026ARK: A Dual-Axis Multimodal Retrieval Benchmark along Reasoning and Knowledge
- 2026SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows
- 2025Look as You Think: Unifying Reasoning and Visual Evidence Attribution for Verifiable Document RAG via Reinforcement Learning
- 2026VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning