mlsanityData to 14 Oct 2026

PROGRESS: Coverage-guided RL to Train Search-augmented LLM Agent

arXiv 2608.00969 ↗First posted 2 Aug 2026cs.AIcited by 0

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 1 more
Abstract (arXiv)

Existing search-augmented LLM agents are trained using Reinforcement Learning to boost its reasoning capabilities. However, these approaches primarily rely on outcome-level rewards, which provide little supervision over search behavior and overlook agent's ability to decompose complex queries properly. To mitigate this issue, we propose PROGRESS which utilizes teacher-guided coverage reward to explicitly shape decomposed query generation of the policy model. During training, frozen teacher models are used to decompose complex queries into essential search queries. These essential search queries are utilized to guide the search behavior of the policy model. Integrated into an R1-style training framework, our approach provides lightweight guidance over query decomposition decisions without dense process-level supervision. Experiments show that coverage-guided RL improves overall task performance, highlighting the importance of explicitly supervising search strategies in agentic LLMs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 21 of its 23 references that are in the map. First posted on arXiv 2 Aug 2026.

Builds on

21 of 23 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025FrugalRAG: Less is More in RL Finetuning for Multi-Hop Question Answering10 Jul
  2. 2025Search Wisely: Mitigating Sub-optimal Agentic Searches By Reducing Uncertainty22 May
  3. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning12 Mar
  4. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning7 Mar
  5. 2025DeepRAG: Thinking to Retrieve Step by Step for Large Language Models3 Feb
  6. 2025Chain-of-Retrieval Augmented Generation24 Jan
  7. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions20 Dec

Other references · 14

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  7. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  8. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  9. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  10. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  11. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  12. 2025ZeroSearch: Incentivize the Search Capability of LLMs without Searchingcited by 208
2 more
  1. 2025HiPRAG: Hierarchical Process Rewards for Efficient Agentic Retrieval Augmented Generationcited by 10
  2. 2025Beyond the limitation of a single query: Train your LLM for query expansion with Reinforcement Learningcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025An Empirical Study on Reinforcement Learning for Reasoning-Search Interleaved LLM Agents
  2. 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications
  3. 2026RAPO: Expanding Exploration for LLM Agents via Retrieval-Augmented Policy Optimization
  4. 2025Search Self-play: Pushing the Frontier of Agent Capability without Supervision
  5. 2026KARL: Knowledge Agents via Reinforcement Learning
  6. 2025DSPO: Stable and Efficient Policy Optimization for Agentic Search and Reasoning
  7. 2025Guiding Exploration in Reinforcement Learning Through LLM-Augmented Observations
  8. 2025SSRL: Self-Search Reinforcement Learning