mlsanityData to 7 Oct 2026

BAPO: Boundary-Aware Policy Optimization for Reliable Agentic Search

arXiv 2601.11037 ↗First posted 16 Jan 2026cs.AIcited by 3

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

RL-based agentic search enables LLMs to solve complex questions via dynamic planning and external search. While this approach significantly enhances accuracy with agent policies optimized via large-scale reinforcement learning, we identify a critical gap in reliability: these agents fail to recognize their reasoning boundaries and rarely admit ``I DON'T KNOW'' (IDK) even when evidence is insufficient or reasoning reaches its limit. The lack of reliability often leads to plausible but unreliable answers, introducing significant risks in many real-world scenarios. To this end, we propose Boundary-Aware Policy Optimization (BAPO), a novel RL framework designed to cultivate reliable boundary awareness without compromising accuracy. BAPO introduces two key components: (i) a group-based boundary-aware reward that encourages an IDK response only when the reasoning reaches its limit, and (ii) an adaptive reward modulator that strategically suspends this reward during early exploration, preventing the model from exploiting IDK as a shortcut. Extensive experiments on four benchmarks demonstrate that BAPO substantially enhances the overall reliability of agentic search.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 29 of its 45 references that are in the map. First posted on arXiv 16 Jan 2026.

Builds on

29 of 45 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Cognitive Kernel-Pro: A Framework for Deep Research Agents and Agent Foundation Models Training1 Aug

Other references · 28

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  7. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  8. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  9. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  10. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  11. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  12. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
16 more
  1. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
  2. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Researchcited by 165
  3. 2023Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthinesscited by 121
  4. 2024Sufficient Context: A New Lens on Retrieval Augmented Generation Systemscited by 69
  5. 2025Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learningcited by 67
  6. 2024Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedbackcited by 64
  7. 2025LinearRAG: Linear Graph Retrieval Augmented Generation on Large-scale Corporacited by 47
  8. 2024Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Modelscited by 34
  9. 2024RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewardscited by 33
  10. 2024Teaching Large Language Models to Express Knowledge Boundary from Their Own Signalscited by 32
  11. 2025Enhancing LLM Reliability via Explicit Knowledge Boundary Modelingcited by 13
  12. 2025Divide-Then-Align: Honest Alignment based on the Knowledge Boundary of RAGcited by 13
  13. 2025Mol-R1: Towards Explicit Long-CoT Reasoning in Molecule Discoverycited by 11
  14. 2025CLPO: Curriculum Learning meets Policy Optimization for LLM Reasoningcited by 11
  15. 2025BARREL: Boundary-Aware Reasoning for Factual and Reliable LRMscited by 8
  16. 2025Alignment through Meta-Weighted Online Sampling: Bridging the Gap between Data Generation and Preference Optimizationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026TSPO: Breaking the Double Homogenization Dilemma in Multi-turn Search Policy Optimizationcited by 5
  2. 2026SAAS: Self-Aware Reinforcement Learning for Over-Search Mitigation in Agentic Searchcited by 2
  3. 2026REIN: Bridging the Gap between Reasoning and Reliability via Reflection and Abstention Alignmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025DSPO: Stable and Efficient Policy Optimization for Agentic Search and Reasoning
  2. 2026Learning to Explore: Scaling Agentic Reasoning via Exploration-Aware Policy Optimization
  3. 2026AIPO: Learning to Reason from Active Interaction
  4. 2026KbSD: Knowledge Boundary aware Self-Distillation for Behavioral Calibration in Agentic Search
  5. 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications
  6. 2025Agentic Reinforced Policy Optimization
  7. 2026RAPO: Expanding Exploration for LLM Agents via Retrieval-Augmented Policy Optimization
  8. 2026Contextual Information Policy Optimization for Search Agents