Learning to Extract Rational Evidence via Reinforcement Learning for Retrieval-Augmented Generation
On the map: Retrieval-augmented generation → Agentic RAG, agents and memory
Abstract (arXiv)
Retrieval-Augmented Generation (RAG) effectively improves the accuracy of Large Language Models (LLMs). However, retrieval noises significantly undermine the quality of LLMs' generation, necessitating the development of denoising mechanisms. Previous works extract evidence straightforwardly without deep thinking, which may risk filtering out key clues and struggle with generalization. To this end, we propose EviOmni, which learns to extract rational evidence via reasoning first and then extracting. Specifically, EviOmni integrates evidence reasoning and evidence extraction into one unified trajectory, followed by knowledge token masking to avoid information leakage, optimized via on-policy reinforcement learning with verifiable rewards in terms of answer, length, and format. Extensive experiments on five benchmark datasets show the superiority of EviOmni, which provides compact and high-quality evidence, enhances the accuracy of downstream tasks, and supports both traditional and agentic RAG systems.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation. Builds on 36 of its 58 references that are in the map. First posted on arXiv 21 Jul 2025.
Builds on
36 of 58 references.
References · 36
- 2017Attention Is All You Needcited by 60,361
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
- 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
- 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
- 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
24 more
- 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
- 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
- 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
- 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
- 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
- 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
- 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 348
- 2025ZeroSearch: Incentivize the Search Capability of LLMs without Searchingcited by 208
- 2024LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMscited by 108
- 2024Reinforcement Learning Enhanced LLMs: A Surveycited by 71
- 2025Think Only When You Need with Large Hybrid-Reasoning Modelscited by 66
- 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agentscited by 51
- 2022An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Taskscited by 36
- 2025KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Modelcited by 36
- 2025RAG-RL: Advancing Retrieval-Augmented Generation via RL and Curriculum Learningcited by 30
- 2021Rider: Reader-Guided Passage Reranking for Open-Domain Question Answeringcited by 28
- 2022Few-shot Reranking for Multi-hop QA via Language Model Promptingcited by 19
- 2024FunnelRAG: A Coarse-to-Fine Progressive Retrieval Paradigm for RAGcited by 18
- 2024An Information Bottleneck Perspective for Effective Noise Filtering on Retrieval-Augmented Generationcited by 16
- 2025APT: Improving Specialist LLM Performance with Weakness Case Acquisition and Iterative Preference Trainingcited by 12
- 2025Re-ranking the Context for Multimodal Retrieval Augmented Generationcited by 11
- 2025Beyond Chunking: Discourse-Aware Hierarchical Retrieval for Long Document Question Answeringcited by 7
- 2024Medico: Towards Hallucination Detection and Correction with Multi-source Evidence Fusioncited by 3
- 2023Make Your Decision Convincing! A Unified Two-Stage Framework: Self-Attribution and Decision-Makingcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Towards Transparent RAG: Fostering Evidence Traceability in LLM Generation via Reinforcement Learning
- 2026Rethinking the Reranker: Boundary-Aware Evidence Selection for Robust Retrieval-Augmented Generation
- 2026GRACE: Reinforcement Learning for Grounded Response and Abstention under Contextual Evidence
- 2026OThink-SRR1: Search, Refine and Reasoning with Reinforced Learning for Large Language Models
- 2026Reinforcement Learning for Large Language Model Selective Evidence Adoption from Contaminated Retrieval Results
- 2026Stateful Evidence-Driven Retrieval-Augmented Generation with Iterative Reasoning
- 2026DynaKRAG: A Unified Framework for Learnable Evidence Control in Multi-Hop Retrieval-Augmented Generation
- 2026RAG-Stress: Probing the Limits of Evidence Reliance in Retrieval-Augmented Generation