mlsanityData to 7 Oct 2026

Search Wisely: Mitigating Sub-optimal Agentic Searches By Reducing Uncertainty

arXiv 2505.17281 ↗First posted 22 May 2025cs.CL cs.AIcited by 16

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 2 more
Abstract (arXiv)

Agentic Retrieval-Augmented Generation (RAG) systems enhance Large Language Models (LLMs) by enabling dynamic, multi-step reasoning and information retrieval. However, these systems often exhibit sub-optimal search behaviors like over-search (retrieving redundant information) and under-search (failing to retrieve necessary information), which hinder efficiency and reliability. This work formally defines and quantifies these behaviors, revealing their prevalence across multiple QA datasets and agentic RAG systems (e.g., one model could have avoided searching in 27.7% of its search steps). Furthermore, we demonstrate a crucial link between these inefficiencies and the models' uncertainty regarding their own knowledge boundaries, where response accuracy correlates with model's uncertainty in its search decisions. To address this, we propose β-GRPO, a reinforcement learning-based training method that incorporates confidence threshold to reward high-certainty search decisions. Experiments on seven QA benchmarks show that β-GRPO enable a 3B model with better agentic RAG ability, outperforming other strong baselines with a 4% higher average exact match score.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 14 of its 21 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

14 of 21 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning12 Mar
  2. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Models9 Jan
  3. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions20 Dec

Other references · 11

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  6. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  7. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  8. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  9. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  10. 2025SMART: Self-Aware Agent for Tool Overuse Mitigationcited by 52
  11. 2024SMARTCAL: An Approach to Self-Aware Tool-Use Evaluation and Calibrationcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 12.

16 papers.

Sort:

By area:

  1. 2025A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challengescited by 58
  2. 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applicationscited by 21
  3. 2025Toward Effective Tool-Integrated Reasoning via Self-Evolved Preference Learningcited by 17
  4. 2026Search-P1: Path-Centric Reward Shaping for Stable and Efficient Agentic RAG Trainingcited by 15
  5. 2025HiPRAG: Hierarchical Process Rewards for Efficient Agentic Retrieval Augmented Generationcited by 9
  6. 2026ET-Agent: Incentivizing Effective Tool-Integrated Reasoning Agent via Behavior Calibrationcited by 4
  7. 2026SAAS: Self-Aware Reinforcement Learning for Over-Search Mitigation in Agentic Searchcited by 2
  8. 2026HALT: Verification-Aware Stopping for Retrieval-Augmented Search Agentscited by 2
  9. 2026AutoSearch: Adaptive Search Depth for Efficient Agentic RAG via Reinforcement Learningcited by 1
  10. 2026Efficient Agentic Reinforcement Learning with On-Policy Intrinsic Knowledge Boundary Enhancementcited by 1
  11. 2026Confidence Laundering in Agent Systems: Why Uncertainty Needs a Latent Carriercited by 1
  12. 2026COMPASS: Cognitive MCTS-Guided Process Alignment for Safe Search Agentscited by 0
4 more
  1. 2026PROGRESS: Coverage-guided RL to Train Search-augmented LLM Agentcited by 0
  2. 2026Search-GRT: Guided Retrieval Training of Search Agents to Optimize for Complex Question Answeringcited by 0
  3. 2026Search-G1: Grounded Search Agents via Representation-Based Intrinsic Rewardscited by 0
  4. 2026MetaRAG: Belief-Action Aligned Policy Optimization for Agentic RAGcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MARAG-R1: Beyond Single Retriever via Reinforcement-Learned Multi-Tool Agentic Retrieval
  2. 2026GRASP: GRanularity-Aware Search Policy for Agentic RAG
  3. 2025QAgent: A modular Search Agent with Interactive Query Understanding
  4. 2026OThink-SRR1: Search, Refine and Reasoning with Reinforced Learning for Large Language Models
  5. 2026A-RAG: Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfaces
  6. 2026SE-Search: Self-Evolving Search Agent via Memory and Dense Reward
  7. 2025Curriculum Guided Reinforcement Learning for Efficient Multi Hop Retrieval Augmented Generation
  8. 2025C-3PO: Compact Plug-and-Play Proxy Optimization to Achieve Human-like Retrieval-Augmented Generation