Data to 7 Oct 2026

VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning

arXiv 2505.22019 ↗First posted 28 May 2025cs.CL cs.AI cs.CVcited by 68

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

Effectively retrieving, reasoning and understanding visually rich information remains a challenge for RAG methods. Traditional text-based methods cannot handle visual-related information. On the other hand, current vision-based RAG approaches are often limited by fixed pipelines and frequently struggle to reason effectively due to the insufficient activation of the fundamental capabilities of models. As RL has been proven to be beneficial for model reasoning, we introduce VRAG-RL, a novel RL framework tailored for complex reasoning across visually rich information. With this framework, VLMs interact with search engines, autonomously sampling single-turn or multi-turn reasoning trajectories with the help of visual perception tokens and undergoing continual optimization based on these samples. Our approach highlights key limitations of RL in RAG domains: (i) Prior Multi-modal RAG approaches tend to merely incorporate images into the context, leading to insufficient reasoning token allocation and neglecting visual-specific perception; and (ii) When models interact with search engines, their queries often fail to retrieve relevant information due to the inability to articulate requirements, thereby leading to suboptimal performance. To address these challenges, we define an action space tailored for visually rich inputs, with actions including cropping and scaling, allowing the model to gather information from a coarse-to-fine perspective. Furthermore, to bridge the gap between users' original inquiries and the retriever, we employ a simple yet effective reward that integrates query rewriting and retrieval performance with a model-based reward. Our VRAG-RL optimizes VLMs for RAG tasks using specially designed RL strategies, aligning the model with real-world applications. The code is available at https://github.com/Alibaba-NLP/VRAG.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 43 of its 53 references that are in the map. First posted on arXiv 28 May 2025.

Builds on

43 of 53 references.

Closest ancestors · 4

Same direction, newest first

Other references · 39

2017Proximal Policy Optimization Algorithmscited by 14,441
2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
2025Qwen2.5-VL Technical Reportcited by 5,418
2024Qwen2.5 Technical Reportcited by 4,713
2023Improved Baselines with Visual Instruction Tuningcited by 4,362
2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371

27 more

2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 466
2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 423
2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 357
2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 307
2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 267
2024Reinforcement learningcited by 236
2024ColPali: Efficient Document Retrieval with Vision Language Modelscited by 190
2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 175
2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 174
2025WebWalker: Benchmarking LLMs in Web Traversalcited by 171
2025ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learningcited by 152
2020PP-OCR: A Practical Ultra Lightweight OCR Systemcited by 147
2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAGcited by 122
2025Perception-R1: Pioneering Perception Policy with Reinforcement Learningcited by 116
2024MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Enginescited by 98
2024Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Modelscited by 97
2024NV-Retriever: Improving text embedding models with effective hard-negative miningcited by 83
2024MindSearch: Mimicking Human Minds Elicits Deep AI Searchercited by 65
2025DeepRetrieval: Hacking Real Search Engines and Retrievers with Large Language Models via Reinforcement Learningcited by 62
2024RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language Modelscited by 58
2025Introducing Visual Perception Token into Multimodal Large Language Modelcited by 26
2025AttentionRAG: Attention-Guided Context Pruning in Retrieval-Augmented Generationcited by 19
2025Unfolding the Headline: Iterative Self-Questioning for News Retrieval and Timeline Summarizationcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 25; 2026: 43.

68 papers.

Sort:

By area:

2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 193
2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 119
2025Tree Search for LLM Agent Reinforcement Learningcited by 73
2025Deep Research: A Survey of Autonomous Research Agentscited by 63
2025A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challengescited by 58
2025Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedbackcited by 31
2026Vision-DeepResearch Benchmark: Rethinking Visual and Textual Search for Multimodal Large Language Modelscited by 30
2026UniCorn: Towards Self-Improving Unified Multimodal Models through Self-Generated Supervisioncited by 23
2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applicationscited by 21
2025A Survey on Agentic Multimodal Large Language Modelscited by 19
2025UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAGcited by 18
2025Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understandingcited by 18

38 more

2026MM-DeepResearch: A Simple and Effective Multimodal Agentic Search Baselinecited by 18
2026GlimpRouter: Efficient Collaborative Inference by Glimpsing One Token of Thoughtscited by 16
2025DocLens : A Tool-Augmented Multi-Agent Framework for Long Visual Document Understandingcited by 15
2025LastingBench: Defend Benchmarks Against Knowledge Leakagecited by 14
2026LensWalk: Agentic Video Understanding by Planning How You See in Videoscited by 14
2025Reinforcement Learning Foundations for Deep Research Systems: A Surveycited by 13
2025Agentic Jigsaw Interaction Learning for Enhancing Visual Perception and Reasoning in Vision-Language Modelscited by 11
2025VR-Thinker: Boosting Video Reward Models through Thinking-with-Image Reasoningcited by 11
2025DualVLA: Building a Generalizable Embodied Agent via Partial Decoupling of Reasoning and Actioncited by 11
2026Thinking in Uncertainty: Mitigating Hallucinations in MLRMs with Latent Entropy-Aware Decodingcited by 11
2025Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understandingcited by 9
2025Reinforcement Learning for Large Model: A Surveycited by 8
2026Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrievalcited by 8
2026PatchCue: Enhancing Vision-Language Model Reasoning with Patch-Based Visual Cuescited by 8
2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
2025M2IO-R1: An Efficient RL-Enhanced Reasoning Framework for Multimodal Retrieval Augmented Multimodal Generationcited by 7
2025Self-Improvement in Multimodal Large Language Models: A Surveycited by 6
2025A Benchmark for Omni-Modal Reasoning in Long Videoscited by 5
2026PaperSearchQA: Learning to Search and Reason over Scientific Papers with RLVRcited by 5
2026VimRAG: Navigating Massive Visual Context in Retrieval-Augmented Generation via Multimodal Memory Graphcited by 5
2026AgenticOCR: Parsing Only What You Need for Efficient Retrieval-Augmented Generationcited by 5
2026CLIPO: Contrastive Learning in Policy Optimization Generalizes RLVRcited by 5
2026SMMBench: A Benchmark for Source-Distributed Multimodal Agent Memorycited by 5
2025Training Multi-Image Vision Agents via End2End Reinforcement Learningcited by 4
2026RetouchIQ: MLLM Agents for Instruction-Based Image Retouching with Generalist Rewardcited by 4
2026Doc-V*:Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQAcited by 4
2025Fed-SE: Federated Self-Evolution for Privacy-Constrained Multi-Environment LLM Agentscited by 3
2026Lang2Act: Fine-Grained Visual Reasoning through Self-Emergent Linguistic Toolchainscited by 3
2026GeoBrowse: A Geolocation Benchmark for Agentic Tool Use with Expert-Annotated Reasoning Tracescited by 3
2026ReAlign: Optimizing the Visual Document Retriever with Reasoning-Guided Fine-Grained Alignmentcited by 3
2026AsyncTool: Evaluating the Asynchronous Function Calling Capability under Multi-Task Scenarioscited by 3
2026VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoningcited by 2
2026DocArena: Turning Raw Documents into Controllable Training Environments for Document Search Agentscited by 2
2025CMRAG: Co-modality-based visual document retrieval and question answeringcited by 1
2025Proactive Reasoning-with-Retrieval Framework for Medical Multimodal Large Language Modelscited by 1
2026When RAG Hurts: Diagnosing and Mitigating Attention Distraction in Retrieval-Augmented LVLMscited by 1
2026Internalizing Meta-Experience into Memory for Guided Reinforcement Learning in Large Language Modelscited by 1
2026POINTS-Seeker: An Open Recipe for Multimodal Search Agents with Visual Memory Managementcited by 1

50 most cited of 68 All 68 (JSON)

Similar, not cited

Close in wording, no citation either way