# AutoSearch: Adaptive Search Depth for Efficient Agentic RAG via Reinforcement Learning

arXiv: https://arxiv.org/abs/2604.17337 · HTML: https://mlsanity.com/paper/2604.17337/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation.
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 19 Apr 2026.

## Placed in

- Retrieval-augmented generation / [Agentic RAG, agents and memory](https://mlsanity.com/rag/agentic-rag-agents-and-memory/)
- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 22 of 22)

- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [Measuring and Narrowing the Compositionality Gap in Language Models](https://mlsanity.com/paper/2210.03350/) (2022)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [ToolRL: Reward is All Tool Learning Needs](https://mlsanity.com/paper/2504.13958/) (2025)
- [Generate rather than Retrieve: Large Language Models are Strong Context Generators](https://mlsanity.com/paper/2209.10063/) (2022)
- [SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning](https://mlsanity.com/paper/2506.19767/) (2025)
- [Acting Less is Reasoning More! Teaching Model to Act Efficiently](https://mlsanity.com/paper/2504.14870/) (2025)
- [Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RL](https://mlsanity.com/paper/2505.10832/) (2025)
- [ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewarding](https://mlsanity.com/paper/2501.07861/) (2025)
- [Promoting Efficient Reasoning with Verifiable Stepwise Reward](https://mlsanity.com/paper/2508.10293/) (2025)
- [Search Wisely: Mitigating Sub-optimal Agentic Searches By Reducing Uncertainty](https://mlsanity.com/paper/2505.17281/) (2025)
- [SMARTCAL: An Approach to Self-Aware Tool-Use Evaluation and Calibration](https://mlsanity.com/paper/2412.12151/) (2024)
- [Perception-Consistency Multimodal Large Language Models Reasoning via Caption-Regularized Policy Optimization](https://mlsanity.com/paper/2509.21854/) (2025)
- [RLAE: Reinforcement Learning-Assisted Ensemble for LLMs](https://mlsanity.com/paper/2506.00439/) (2025)
- [Videos are Sample-Efficient Supervisions: Behavior Cloning from Videos via Latent Representations](https://mlsanity.com/paper/2512.21586/) (2025)
- [Saliency-Guided Representation with Consistency Policy Learning for Visual Unsupervised Reinforcement Learning](https://mlsanity.com/paper/2604.05931/) (2026)

## Built on by (most cited first, 1 of 1)

- [The Missing Complement: State-Conditioned Minimal Sufficient Evidence for Coding Agents](https://mlsanity.com/paper/2609.20050/) (2026)

## Abstract (arXiv)

Agentic retrieval-augmented generation (RAG) systems enable large language models (LLMs) to solve complex tasks through multi-step interaction with external retrieval tools. However, such multi-step interaction often involves redundant search steps, incurring substantial computational cost and latency. Prior work limits search depth (i.e., the number of search steps) to reduce cost, but this often leads to underexploration of complex questions. To address this, we first investigate how search depth affects accuracy and find a minimal sufficient search depth that defines an accuracy-efficiency trade-off, jointly determined by question complexity and the agent's capability. Furthermore, we propose AutoSearch, a reinforcement learning (RL) framework that evaluates each search step via self-generated intermediate answers. By a self-answering mechanism, AutoSearch identifies the minimal sufficient search depth and promotes efficient search by rewarding its attainment while penalizing over-searching. In addition, reward mechanisms are introduced to stabilize search behavior and improve answer quality on complex questions. Extensive experiments on multiple benchmarks show that AutoSearch achieves a superior accuracy-efficiency trade-off, alleviating over-searching while preserving search quality.