mlsanityData to 7 Oct 2026

V-Retrver: Evidence-Driven Agentic Reasoning for Universal Multimodal Retrieval

arXiv 2602.06034 ↗First posted 5 Feb 2026cs.CVcited by 2

On the map: LLM reasoning → Multimodal reasoning

also in 2 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have recently been applied to universal multimodal retrieval, where Chain-of-Thought (CoT) reasoning improves candidate reranking. However, existing approaches remain largely language-driven, relying on static visual encodings and lacking the ability to actively verify fine-grained visual evidence, which often leads to speculative reasoning in visually ambiguous cases. We propose V-Retrver, an evidence-driven retrieval framework that reformulates multimodal retrieval as an agentic reasoning process grounded in visual inspection. V-Retrver enables an MLLM to selectively acquire visual evidence during reasoning via external visual tools, performing a multimodal interleaved reasoning process that alternates between hypothesis generation and targeted visual verification.To train such an evidence-gathering retrieval agent, we adopt a curriculum-based learning strategy combining supervised reasoning activation, rejection-based refinement, and reinforcement learning with an evidence-aligned objective. Experiments across multiple multimodal retrieval benchmarks demonstrate consistent improvements in retrieval accuracy (with 23.0% improvements on average), perception-driven reasoning reliability, and generalization.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 78 of its 87 references that are in the map. First posted on arXiv 5 Feb 2026.

Builds on

78 of 87 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025AdaTooler-V: Adaptive Tool-Use for Images and Videos18 Dec
  2. 2025Knowing the Answer Isn't Enough: Fixing Reasoning Path Failures in LVLMs6 Dec
  3. 2025Retrv-R1: A Reasoning-Driven MLLM Framework for Universal and Efficient Multimodal Retrieval3 Oct
  4. 2025Vision-EKIPL: External Knowledge-Infused Policy Learning for Visual Reasoning7 Jun
  5. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
  6. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
  7. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
  8. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec

Other references · 70

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2025Qwen2.5-VL Technical Reportcited by 5,418
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  8. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  9. 2023Qwen Technical Reportcited by 3,273
  10. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  11. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  12. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
38 more
  1. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  2. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  3. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  4. 2025s1: Simple test-time scalingcited by 1,311
  5. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  6. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  7. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  8. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  9. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677
  10. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 545
  11. 2016Visual Dialogcited by 377
  12. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 285
  13. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 203
  14. 2024LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Imagescited by 193
  15. 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 188
  16. 2016Visual Storytellingcited by 176
  17. 2023Zero-Shot Composed Image Retrieval with Textual Inversioncited by 171
  18. 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 166
  19. 2023UniIR: Training and Benchmarking Universal Multimodal Information Retrieverscited by 165
  20. 2025LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Tokencited by 164
  21. 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 159
  22. 2023Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrievalcited by 155
  23. 2021Probabilistic Embeddings for Cross-Modal Retrievalcited by 150
  24. 2024E5-V: Universal Embeddings with Multimodal Large Language Modelscited by 123
  25. 2024MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructionscited by 119
  26. 2023Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entitiescited by 112
  27. 2022Task-aware Retrieval with Instructionscited by 107
  28. 2024Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Expertscited by 107
  29. 2024LamRA: Large Multimodal Model as Your Advanced Retrieval Assistantcited by 106
  30. 2023Encyclopedic VQA: Visual questions about detailed properties of fine-grained categoriescited by 105
  31. 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performancecited by 94
  32. 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Usecited by 92
  33. 2024MegaPairs: Massive Data Synthesis For Universal Multimodal Retrievalcited by 84
  34. 2023CLIP for All Things Zero-Shot Sketch-Based Image Retrieval, Fine-Grained or Notcited by 79
  35. 2025Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMscited by 73
  36. 2022Universal Vision-Language Dense Retrieval: Learning A Unified Representation Space for Multi-Modal Retrievalcited by 57
  37. 2025LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learningcited by 54
  38. 2023Fine-grained Late-interaction Multi-modal Retrieval for Retrieval Augmented Visual Question Answeringcited by 51

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026VimRAG: Navigating Massive Visual Context in Retrieval-Augmented Generation via Multimodal Memory Graph
  2. 2026Visual-Seeker: Towards Visual-Native Multimodal Agentic Search via Active Visual Reasoning
  3. 2026Reasoning-Augmented Representations for Multimodal Retrieval
  4. 2026MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory
  5. 2026ARK: A Dual-Axis Multimodal Retrieval Benchmark along Reasoning and Knowledge
  6. 2026SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows
  7. 2025Look as You Think: Unifying Reasoning and Visual Evidence Attribution for Verifiable Document RAG via Reinforcement Learning
  8. 2026VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning