IterSynth: Rethinking Deep Search Agents via Role-Decoupled Iterative Synthesis
On the map: LLM reasoning → Reinforcement learning for reasoning also fits: Reasoning for agents, planning and code
also in 1 more
Abstract (arXiv)
Deep search requires LLM agents to decompose complex queries, search for evidence, and synthesize grounded answers, yet existing ReAct-style agents suffer from two limitations: role coupling, where one policy must handle planning, evidence use, and synthesis; and context accumulation, where growing search histories introduce noise and obscure useful information. To address these issues, we propose IterSynth, a role-decoupled and summary-based paradigm that alternates between a Planner for identifying information needs and a Synthesizer for integrating evidence into an evolving summary state. This design separates planning from synthesis while using the summary as the persistent state of search, reducing both capability coupling and context noise. To train IterSynth effectively, we further introduce Role-Decoupled Policy Optimization (RDPO) for reinforcement learning, which combines terminal outcome rewards with turn-level rubric evaluations and computes role-specific advantages for more precise credit assignment. Experiments on five long-horizon deep-search benchmarks such as BrowseComp and Xbench-DS show that IterSynth-8B achieves an average score of 50.7, surpassing the strongest prior ≤8B agent by +4.2%. Moreover, IterSynth serves as a model-agnostic prompting paradigm, delivering substantial zero-shot gains over ReAct and similar prompting paradigms on frontier proprietary models.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 35 references that are in the map. First posted on arXiv 24 Sep 2026.
Builds on
28 of 35 references.
References · 28
- 2025Qwen3 Technical Reportcited by 8,820
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
- 2023GAIA: a benchmark for General AI Assistantscited by 1,128
- 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
- 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
- 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
- 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 467
- 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agentcited by 246
- 2025BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinesecited by 145
16 more
- 2025ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarizationcited by 141
- 2025WebShaper: Agentically Data Synthesizing via Information-Seeking Formalizationcited by 121
- 2025AgentFold: Long-Horizon Web Agents with Proactive Context Managementcited by 103
- 2025WebExplorer: Explore and Evolve for Training Long-Horizon Web Agentscited by 92
- 2025MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scalingcited by 81
- 2025Tongyi DeepResearch Technical Reportcited by 66
- 2025WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learningcited by 61
- 2025DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RLcited by 55
- 2026REDSearcher: A Scalable and Cost-Efficient Framework for Long-Horizon Search Agentscited by 49
- 2025WebWeaver: Structuring Web-Scale Evidence with Dynamic Outlines for Open-Ended Deep Researchcited by 46
- 2026OpenSeeker: Democratizing Frontier Search Agents by Fully Open-Sourcing Training Datacited by 39
- 2025IterResearch: Rethinking Long-Horizon Agents with Interaction Scalingcited by 23
- 2026OffSeeker: Online Reinforcement Learning Is Not All You Need for Deep Research Agentscited by 12
- 2026AgentCPM-Explore: Realizing Long-Horizon Deep Exploration for Edge-Scale Agentscited by 12
- 2025InfoFlow: Reinforcing Search Agent Via Reward Density Optimizationcited by 11
- 2025InfoAgent: Advancing Autonomous Information-Seeking Agentscited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Search More, Think Less: Rethinking Long-Horizon Agentic Search for Efficiency and Generalization
- 2026Argus: Evidence Assembly for Scalable Deep Research Agents
- 2026Beyond Outcome Rewards: Step-Level Self-Distilled Policy Optimization for Deep Search Agents
- 2026SearchArt: Training Long-Horizon Search Agent with Scalable Synthetic and Verified Task
- 2026Chaining the Evidence: Robust Reinforcement Learning for Deep Search Agents with Citation-Aware Rubric Rewards
- 2025HierSearch: A Hierarchical Enterprise Deep Search Framework Integrating Local and Web Searches
- 2025Fathom-DeepResearch: Unlocking Long Horizon Information Retrieval and Synthesis for SLMs
- 2026AREX: Towards a Recursively Self-Improving Agent for Deep Research