mlsanityData to 7 Oct 2026

Search-GRT: Guided Retrieval Training of Search Agents to Optimize for Complex Question Answering

arXiv 2608.00974 ↗First posted 2 Aug 2026cs.AIcited by 0

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

The effective use of search engines by large language models (LLMs) remains a significant challenge, particularly in complex, multi-hop question-answering (MHQA) tasks. These tasks require the model to decompose questions into subqueries, retrieve relevant information, and synthesize answers from multiple sources, often leading to cascading errors due to poor retrieval in early stages. Reinforcement learning (RL) has shown promise in improving LLMs' search capabilities, but it often suffers from sparse rewards during training, hindering the model's ability to learn effectively. To address these challenges, we introduce Guided Retrieval Training (GRT), a novel method that improves the performance of a search agent by restricting the retrieval process during RL training using ground truth information. By focusing on a curated set of relevant documents, GRT provides the model with a stronger learning signal, mitigating the problem of sparse rewards and improving its ability to generate accurate subqueries and synthesize correct answers. Our experimental results demonstrate that GRT achieves consistent performance improvements over existing methods, such as Search-R1, across a wide range of question-answering (QA) tasks. Notably, GRT excels in MHQA tasks, achieving over 40% improvements in performance. Additionally, GRT enhances training efficiency by achieving better QA performance with fewer training steps.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 31 of its 34 references that are in the map. First posted on arXiv 2 Aug 2026.

Builds on

31 of 34 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025HiPRAG: Hierarchical Process Rewards for Efficient Agentic Retrieval Augmented Generation9 Oct
  2. 2025Search Wisely: Mitigating Sub-optimal Agentic Searches By Reducing Uncertainty22 May
  3. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Models9 Jan

Other references · 28

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  10. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  11. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  12. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
16 more
  1. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  2. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  3. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  4. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  5. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  6. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  7. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  8. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 417
  9. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 322
  10. 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 211
  11. 2025ZeroSearch: Incentivize the Search Capability of LLMs without Searchingcited by 203
  12. 2023Enhancing Conversational Search: Large Language Model-Aided Informative Query Rewritingcited by 69
  13. 2025Chain-of-Retrieval Augmented Generationcited by 48
  14. 2025DeepRAG: Thinking to Retrieve Step by Step for Large Language Modelscited by 37
  15. 2025Search Arena: Analyzing Search-Augmented LLMscited by 30
  16. 2024GenRewrite: Query Rewriting via Large Language Modelscited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025StepSearch: Igniting LLMs Search Ability via Step-Wise Proximal Policy Optimization
  2. 2026PRISMA: Reinforcement Learning Guided Two-Stage Policy Optimization in Multi-Agent Architecture for Open-Domain Multi-Hop Question Answering
  3. 2025SEM: Reinforcement Learning for Search-Efficient Large Language Models
  4. 2026Plan Before Search: Search Agents Need Plan
  5. 2025O2-Searcher: A Searching-based Agent Model for Open-Domain Open-Ended Question Answering
  6. 2025Curriculum Guided Reinforcement Learning for Efficient Multi Hop Retrieval Augmented Generation
  7. 2025Beyond the limitation of a single query: Train your LLM for query expansion with Reinforcement Learning
  8. 2026OThink-SRR1: Search, Refine and Reasoning with Reinforced Learning for Large Language Models