mlsanityData to 14 Oct 2026

SAAS: Self-Aware Reinforcement Learning for Over-Search Mitigation in Agentic Search

arXiv 2605.29796 ↗First posted 28 May 2026cs.AI cs.CL cs.LGcited by 2

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Agentic search enables LLMs to solve complex multi-hop questions through iterative reasoning and external search. Despite the effectiveness, these systems often suffer from a critical limitation in practice: agents fail to recognize their own knowledge boundaries, blindly triggering searches when internal knowledge suffices and failing to terminate search even when adequate evidence has been collected. The lack of self-awareness leads to severe over-search, incurring substantial inference latency and prohibitive computational cost. To this end, we propose SAAS, a novel RL framework designed to cultivate dynamic self-awareness that precisely regulates search behavior without compromising accuracy. SAAS introduces three key components: (i) a search boundary modeling mechanism, which identifies the search boundary under the evolving policy by contrasting search-disabled and search-enabled rollouts; (ii) a boundary-aware reward module, which translates this boundary awareness into trajectory-level penalties, suppressing unnecessary and redundant searches; and (iii) a stage-wise optimization strategy, which leverages a sequential curriculum to prioritize reasoning over search regularization, thereby avoiding reward hacking. Extensive experiments demonstrate that SAAS substantially reduces over-search, while maintaining accuracy. Our code and implementation details are released at https://github.com/XMUDeepLIT/SAAS.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 30 of its 58 references that are in the map. First posted on arXiv 28 May 2026.

Builds on

30 of 58 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026BAPO: Boundary-Aware Policy Optimization for Reliable Agentic Search16 Jan
  2. 2025MaskSearch: A Universal Pre-Training Framework to Enhance Agentic Search Capability26 May
  3. 2025Search Wisely: Mitigating Sub-optimal Agentic Searches By Reducing Uncertainty22 May
  4. 2025s3: You Don't Need That Much Data to Train a Search Agent via RL20 May
  5. 2025Reinforced Internal-External Knowledge Synergistic Reasoning for Efficient Adaptive Search Agent12 May
  6. 2025Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG15 Jan

Other references · 24

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  4. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  5. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  6. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  7. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  8. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  9. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  10. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  11. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  12. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
12 more
  1. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 392
  2. 2025ToolRL: Reward is All Tool Learning Needscited by 362
  3. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 335
  4. 2025StepSearch: Igniting LLMs Search Ability via Step-Wise Proximal Policy Optimizationcited by 88
  5. 2025LinearRAG: Linear Graph Retrieval Augmented Generation on Large-scale Corporacited by 49
  6. 2025ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewardingcited by 32
  7. 2025Beyond Correctness: Harmonizing Process and Outcome Rewards through RL Trainingcited by 29
  8. 2025Promoting Efficient Reasoning with Verifiable Stepwise Rewardcited by 28
  9. 2025You Don't Need Pre-built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structurescited by 23
  10. 2024CtrlA: Adaptive Retrieval-Augmented Generation via Inherent Controlcited by 13
  11. 2025Reliable Reasoning Path: Distilling Effective Guidance for LLM Reasoning with Knowledge Graphscited by 9
  12. 2025SUGAR: Leveraging Contextual Confidence for Smarter Retrievalcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligencecited by 3
  2. 2026Search-G1: Grounded Search Agents via Representation-Based Intrinsic Rewardscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications
  2. 2025ReSeek: A Self-Correcting Framework for Search Agents with Instructive Rewards
  3. 2025SSRL: Self-Search Reinforcement Learning
  4. 2026SearchGym: Bootstrapping Real-World Search Agents via Cost-Effective and High-Fidelity Environment Simulation
  5. 2026SelfSearch: Reward-Free Search for Self-Improving Agents
  6. 2026KARL: Knowledge Agents via Reinforcement Learning
  7. 2026AutoSearch: Adaptive Search Depth for Efficient Agentic RAG via Reinforcement Learning
  8. 2026To Answer or to Abstain: Mitigating Search-Agent Hallucinations via Abstention-Aware Reinforcement Learning