mlsanityData to 14 Oct 2026

Q-RAG: Long Context Multi-step Retrieval via Value-based Embedder Training

arXiv 2511.07328 ↗First posted 10 Nov 2025cs.LG cs.IRcited by 4

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) methods enhance LLM performance by efficiently filtering relevant context for LLMs, reducing hallucinations and inference cost. However, most existing RAG methods focus on single-step retrieval, which is often insufficient for answering complex questions that require multi-step search. Recently, multi-step retrieval approaches have emerged, typically involving the fine-tuning of small LLMs to perform multi-step retrieval. This type of fine-tuning is highly resource-intensive and does not enable the use of larger LLMs. In this work, we propose Q-RAG, a novel approach that fine-tunes the Embedder model for multi-step retrieval using reinforcement learning (RL). Q-RAG offers a competitive, resource-efficient alternative to existing multi-step retrieval methods for open-domain question answering and achieves state-of-the-art results on the popular long-context benchmarks BabiLong and RULER for contexts up to 10M tokens. Code is available at https://github.com/griver/Q-RAG

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 30 of its 39 references that are in the map. First posted on arXiv 10 Nov 2025.

Builds on

30 of 39 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025RAG-RL: Advancing Retrieval-Augmented Generation via RL and Curriculum Learning17 Mar
  2. 2024In Defense of RAG in the Era of Long-Context Language Models3 Sep

Other references · 28

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  5. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  6. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  7. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  8. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 867
  9. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
  10. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
  11. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 433
  12. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 336
16 more
  1. 2024Titans: Learning to Memorize at Test Timecited by 328
  2. 2025Agentic Retrieval-Augmented Generation: A Survey on Agentic RAGcited by 249
  3. 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystackcited by 215
  4. 2022Recurrent Memory Transformercited by 203
  5. 2024An Empirical Study of Mamba-based Language Modelscited by 186
  6. 2025A Comprehensive Survey on Long Context Language Modelingcited by 117
  7. 2024AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agentscited by 102
  8. 2024Simplifying Deep Temporal Difference Learningcited by 78
  9. 2025ATLAS: Learning to Optimally Memorize the Context at Test Timecited by 66
  10. 2024GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Modelscited by 43
  11. 2024IM-RAG: Multi-Round Retrieval-Augmented Generation Through Learning Inner Monologuescited by 40
  12. 2023End-to-End Beam Retrieval for Multi-Hop Question Answeringcited by 35
  13. 2025LongRoPE2: Near-Lossless LLM Context Window Scalingcited by 24
  14. 2024Associative Recurrent Memory Transformercited by 15
  15. 2022Explain My Surprise: Learning Efficient Long-Term Memory by Predicting Uncertain Outcomescited by 11
  16. 2025Survey of Specialized Large Language Modelcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026BitNet Text Embeddingscited by 0
  2. 2026GTA-RAG: Graph-Trajectory-Augmented Reinforcement Learning for Multi-Turn Retrieval-Augmented Reasoningcited by 0
  3. 2026Learning to Route in Visual Space via Multi-Step Embedding Retrievalcited by 0
  4. 2026SEEDS: Preview-Guided Evidence Admission for Search Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ELITE: Embedding-Less retrieval with Iterative Text Exploration
  2. 2025Post-training an LLM for RAG? Train on Self-Generated Demonstrations
  3. 2024SFR-RAG: Towards Contextually Faithful LLMs
  4. 2025R3-RAG: Learning Step-by-Step Reasoning and Retrieval for LLMs via Reinforcement Learning
  5. 2024One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Models
  6. 2025Multiple Abstraction Level Retrieve Augment Generation
  7. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers
  8. 2024W-RAG: Weakly Supervised Dense Retrieval in RAG for Open-domain Question Answering