mlsanityData to 7 Oct 2026

Reinforcement Fine-Tuning for History-Aware Dense Retriever in RAG

arXiv 2602.03645 ↗First posted 3 Feb 2026cs.LGcited by 1

On the map: Retrieval-augmented generation → Agentic RAG, agents and memory

Abstract (arXiv)

Retrieval-augmented generation (RAG) enables large language models (LLMs) to produce evidence-based responses, and its performance hinges on the matching between the retriever and LLMs. Retriever optimization has emerged as an efficient alternative to fine-tuning LLMs. However, existing solutions suffer from objective mismatch between retriever optimization and the goal of RAG pipeline. Reinforcement learning (RL) provides a promising solution to address this limitation, yet applying RL to retriever optimization introduces two fundamental challenges: 1) the deterministic retrieval is incompatible with RL formulations, and 2) state aliasing arises from query-only retrieval in multi-hop reasoning. To address these challenges, we replace deterministic retrieval with stochastic sampling and formulate RAG as a Markov decision process, making retriever optimizable by RL. Further, we incorporate retrieval history into the state at each retrieval step to mitigate state aliasing. Extensive experiments across diverse RAG pipelines, datasets, and retriever scales demonstrate consistent improvements of our approach in RAG performance.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation. Builds on 34 of its 42 references that are in the map. First posted on arXiv 3 Feb 2026.

Builds on

34 of 42 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Optimizing Retrieval for RAG via Reinforcement Learning28 Oct
  2. 2025DeepRAG: Thinking to Retrieve Step by Step for Large Language Models3 Feb
  3. 2025Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG15 Jan
  4. 2024RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewards17 Oct

Other references · 30

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  10. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  11. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  12. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
18 more
  1. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  2. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  3. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  4. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  5. 2023A Comprehensive Overview of Large Language Modelscited by 639
  6. 2023Active Retrieval Augmented Generationcited by 506
  7. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  8. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 272
  9. 2025ZeroSearch: Incentivize the Search Capability of LLMs without Searchingcited by 203
  10. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
  11. 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMscited by 119
  12. 2024Bridging the Preference Gap between Retrievers and LLMscited by 69
  13. 2025Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replaycited by 52
  14. 2025Chain-of-Retrieval Augmented Generationcited by 48
  15. 2024M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitionscited by 38
  16. 2024Stochastic RAG: End-to-End Retrieval-Augmented Generation through Expected Utility Maximizationcited by 29
  17. 2024Reward-RAG: Enhancing RAG with Reward Driven Supervisioncited by 7
  18. 2025Retrieval-GRPO: A Multi-Objective Reinforcement Learning Framework for Dense Retrieval in Taobao Searchcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MARAG-R1: Beyond Single Retriever via Reinforcement-Learned Multi-Tool Agentic Retrieval
  2. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
  3. 2025Domain-Aware RAG: MoL-Enhanced RL for Efficient Training and Scalable Retrieval
  4. 2025R3-RAG: Learning Step-by-Step Reasoning and Retrieval for LLMs via Reinforcement Learning
  5. 2026Understanding the Behaviors of Environment-aware Information Retrieval
  6. 2026OThink-SRR1: Search, Refine and Reasoning with Reinforced Learning for Large Language Models
  7. 2025Curriculum Guided Reinforcement Learning for Efficient Multi Hop Retrieval Augmented Generation
  8. 2024Toward Optimal Search and Retrieval for RAG