# Scent of Knowledge: Optimizing Search-Enhanced Reasoning with Information Foraging

arXiv: https://arxiv.org/abs/2505.09316 · HTML: https://mlsanity.com/paper/2505.09316/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 42 of its 53 references that are in the map.
- First posted on arXiv 14 May 2025.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)
- Retrieval-augmented generation / [Agentic RAG, agents and memory](https://mlsanity.com/rag/agentic-rag-agents-and-memory/)

## Builds on (most cited first, 30 of 42)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [Measuring and Narrowing the Compositionality Gap in Language Models](https://mlsanity.com/paper/2210.03350/) (2022)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [MuSiQue: Multihop Questions via Single-hop Question Composition](https://mlsanity.com/paper/2108.00573/) (2021)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Active Retrieval Augmented Generation](https://mlsanity.com/paper/2305.06983/) (2023)
- [Reinforced Self-Training (ReST) for Language Modeling](https://mlsanity.com/paper/2308.08998/) (2023)
- [Search-o1: Agentic Search-Enhanced Large Reasoning Models](https://mlsanity.com/paper/2501.05366/) (2025)
- [Training Language Models to Self-Correct via Reinforcement Learning](https://mlsanity.com/paper/2409.12917/) (2024)
- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search](https://mlsanity.com/paper/2406.03816/) (2024)
- [Large Language Models for Information Retrieval: A Survey](https://mlsanity.com/paper/2308.07107/) (2023)

## Built on by (most cited first, 20 of 20)

- [Open Data Synthesis For Deep Research](https://mlsanity.com/paper/2509.00375/) (2025)
- [InfoFlow: Reinforcing Search Agent Via Reward Density Optimization](https://mlsanity.com/paper/2510.26575/) (2025)
- [Search-R2: Enhancing Search-Integrated Reasoning via Actor-Refiner Collaboration](https://mlsanity.com/paper/2602.03647/) (2026)
- [DeepResearch-9K: A Challenging Benchmark Dataset of Deep-Research Agent](https://mlsanity.com/paper/2603.01152/) (2026)
- [MR2-Bench: Going Beyond Matching to Reasoning in Multimodal Retrieval](https://mlsanity.com/paper/2509.26378/) (2025)
- [How Many Tools Should an LLM Agent See? A Chance-Corrected Answer](https://mlsanity.com/paper/2605.24660/) (2026)
- [Interact-RAG: Reason and Interact with the Corpus, Beyond Black-Box Retrieval](https://mlsanity.com/paper/2510.27566/) (2025)
- [Think Before You Retrieve: Learning Test-Time Adaptive Search with Small Language Models](https://mlsanity.com/paper/2511.07581/) (2025)
- [Self-Induced Outcome Potential: Turn-Level Credit Assignment for Agents without Verifiers](https://mlsanity.com/paper/2605.04984/) (2026)
- [REX-RAG: Reasoning Exploration with Policy Correction in Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.08149/) (2025)
- [Improving Search Agent with One Line of Code](https://mlsanity.com/paper/2603.10069/) (2026)
- [ORBIT: Scalable and Verifiable Data Generation for Search Agents on a Tight Budget](https://mlsanity.com/paper/2604.01195/) (2026)
- [Agentic Conversational Search with Contextualized Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2601.13115/) (2026)
- [InterDeepResearch: Enabling Human-Agent Collaborative Information Seeking through Interactive Deep Research](https://mlsanity.com/paper/2603.12608/) (2026)
- [Caesar: Deep Agentic Web Exploration for Creative Answer Synthesis](https://mlsanity.com/paper/2604.20855/) (2026)
- [Plan Before Search: Search Agents Need Plan](https://mlsanity.com/paper/2605.28354/) (2026)
- [Strategic Decision Support for AI Agents](https://mlsanity.com/paper/2606.12587/) (2026)
- [SearchOS-V1: Towards Robust Open-Domain Information-Seeking Agent Collaboration](https://mlsanity.com/paper/2607.15257/) (2026)
- [AlgoWorlds: Benchmarking Tool Use for Global Optimization in Algorithmic Worlds](https://mlsanity.com/paper/2608.29397/) (2026)
- [SubSearch: Intermediate Rewards for Unsupervised Guided Reasoning in Complex Retrieval](https://mlsanity.com/paper/2604.07415/) (2026)

## Abstract (arXiv)

Augmenting large language models (LLMs) with external retrieval has become a standard method to address their inherent knowledge cutoff limitations. However, traditional retrieval-augmented generation methods employ static, pre-inference retrieval strategies, making them inadequate for complex tasks involving ambiguous, multi-step, or evolving information needs. Recent advances in test-time scaling techniques have demonstrated significant potential in enabling LLMs to dynamically interact with external tools, motivating the shift toward adaptive inference-time retrieval. Inspired by Information Foraging Theory (IFT), we propose InForage, a reinforcement learning framework that formalizes retrieval-augmented reasoning as a dynamic information-seeking process. Unlike existing approaches, InForage explicitly rewards intermediate retrieval quality, encouraging LLMs to iteratively gather and integrate information through adaptive search behaviors. To facilitate training, we construct a human-guided dataset capturing iterative search and reasoning trajectories for complex, real-world web tasks. Extensive evaluations across general question answering, multi-hop reasoning tasks, and a newly developed real-time web QA dataset demonstrate InForage's superior performance over baseline methods. These results highlight InForage's effectiveness in building robust, adaptive, and efficient reasoning agents.