Beyond Monolithic Architectures: A Multi-Agent Search and Knowledge Optimization Framework for Agentic Search
On the map: LLM reasoning → Knowledge- and tool-augmented reasoning
also in 1 more
Abstract (arXiv)
Agentic search has emerged as a promising paradigm for complex information seeking by enabling Large Language Models (LLMs) to interleave reasoning with tool use. However, prevailing systems rely on monolithic agents that suffer from structural bottlenecks, including unconstrained reasoning outputs that inflate trajectories, sparse outcome-level rewards that complicate credit assignment, and stochastic search noise that destabilizes learning. To address these challenges, we propose M-ASK (Multi-Agent Search and Knowledge), a framework that explicitly decouples agentic search into two complementary roles: Search Behavior Agents, which plan and execute search actions, and Knowledge Management Agents, which aggregate, filter, and maintain a compact internal context. This decomposition allows each agent to focus on a well-defined subtask and reduces interference between search and context construction. Furthermore, to enable stable coordination, M-ASK employs turn-level rewards to provide granular supervision for both search decisions and knowledge updates. Experiments on multi-hop QA benchmarks demonstrate that M-ASK outperforms strong baselines, achieving not only superior answer accuracy but also significantly more stable training dynamics.footnoteThe source code for M-ASK is available at https://github.com/chenyiqun/M-ASK.
Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 22 of its 37 references that are in the map. First posted on arXiv 8 Jan 2026.
Builds on
22 of 37 references.
Closest ancestors · 3
Same direction, newest first
- 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning12 Mar
- 2025Search-o1: Agentic Search-Enhanced Large Reasoning Models9 Jan
- 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions20 Dec
Other references · 19
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
- 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
- 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
- 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
- 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
- 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
- 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
- 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
- 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 390
7 more
- 2025Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learningcited by 237
- 2013The Complexity of Decentralized Control of Markov Decision Processescited by 231
- 2024Bridging the Preference Gap between Retrievers and LLMscited by 69
- 2024MindSearch: Mimicking Human Minds Elicits Deep AI Searchercited by 66
- 2025MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End Reinforcement Learningcited by 27
- 2025On Group Relative Policy Optimization Collapse in Agent Search: The Lazy Likelihood-Displacementcited by 15
- 2025Iterative Self-Incentivization Empowers Large Language Models as Agentic Searcherscited by 11
Built on by
Citing papers per year: 2026: 11.
11 papers.
- 2026From Question Answering to Task Completion: A Survey on Agent System and Harness Designcited by 12
- 2026GrepSeek: Training Search Agents for Direct Corpus Interactioncited by 10
- 2026Learning Decentralized LLM Collaboration with Multi-Agent Actor Criticcited by 8
- 2026OASES: Outcome-Aligned Search-Evaluation Co-Training for Agentic Searchcited by 7
- 2026JADE: Bridging the Strategic-Operational Gap in Dynamic Agentic RAGcited by 6
- 2026UnityMAS-O: A General RL Optimization Framework for LLM-Based Multi-Agent Systemscited by 6
- 2026A-MapReduce: Executing Wide Search via Agentic MapReducecited by 5
- 2026MARTI-MARS2: Scaling Multi-Agent Self-Search via Reinforcement Learning for Code Generationcited by 3
- 2026SearchArt: Training Long-Horizon Search Agent with Scalable Synthetic and Verified Taskcited by 3
- 2026Divide and Cooperate: Role-Decomposed Multi-Agent LLM Training with Cross-Agent Learning Signalscited by 2
- 2026MaxKernel: Agentic Kernel Generation for TPUscited by 1
- 2026MaxKernel: Agentic Kernel Generation for TPUs3 Sep 2026
- 2026SearchArt: Training Long-Horizon Search Agent with Scalable Synthetic and Verified Task25 Jul 2026
- 2026From Question Answering to Task Completion: A Survey on Agent System and Harness Design14 Jun 2026
- 2026Divide and Cooperate: Role-Decomposed Multi-Agent LLM Training with Cross-Agent Learning Signals9 Jun 2026
- 2026GrepSeek: Training Search Agents for Direct Corpus Interaction28 May 2026
- 2026UnityMAS-O: A General RL Optimization Framework for LLM-Based Multi-Agent Systems26 May 2026
- 2026OASES: Outcome-Aligned Search-Evaluation Co-Training for Agentic Search4 Apr 2026
- 2026MARTI-MARS2: Scaling Multi-Agent Self-Search via Reinforcement Learning for Code Generation8 Feb 2026
- 2026A-MapReduce: Executing Wide Search via Agentic MapReduce1 Feb 2026
- 2026Learning Decentralized LLM Collaboration with Multi-Agent Actor Critic29 Jan 2026
- 2026JADE: Bridging the Strategic-Operational Gap in Dynamic Agentic RAG29 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2025Supervising the search process produces reliable and generalizable information-seeking agents
- 2025RE-Searcher: Robust Agentic Search with Goal-oriented Planning and Self-reflection
- 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agents
- 2025DynaSearcher: Dynamic Knowledge Graph Augmented Search Agent via Multi-Reward Reinforcement Learning
- 2025Knowledge-Aware Iterative Retrieval for Multi-Agent Systems
- 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications
- 2026REDSearcher: A Scalable and Cost-Efficient Framework for Long-Horizon Search Agents
- 2026Search, Do not Guess: Teaching Small Language Models to Be Effective Search Agents