mlsanityData to 14 Oct 2026

IterSynth: Rethinking Deep Search Agents via Role-Decoupled Iterative Synthesis

arXiv 2609.29444 ↗First posted 24 Sep 2026cs.CL cs.AIcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning also fits: Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Deep search requires LLM agents to decompose complex queries, search for evidence, and synthesize grounded answers, yet existing ReAct-style agents suffer from two limitations: role coupling, where one policy must handle planning, evidence use, and synthesis; and context accumulation, where growing search histories introduce noise and obscure useful information. To address these issues, we propose IterSynth, a role-decoupled and summary-based paradigm that alternates between a Planner for identifying information needs and a Synthesizer for integrating evidence into an evolving summary state. This design separates planning from synthesis while using the summary as the persistent state of search, reducing both capability coupling and context noise. To train IterSynth effectively, we further introduce Role-Decoupled Policy Optimization (RDPO) for reinforcement learning, which combines terminal outcome rewards with turn-level rubric evaluations and computes role-specific advantages for more precise credit assignment. Experiments on five long-horizon deep-search benchmarks such as BrowseComp and Xbench-DS show that IterSynth-8B achieves an average score of 50.7, surpassing the strongest prior ≤8B agent by +4.2%. Moreover, IterSynth serves as a model-agnostic prompting paradigm, delivering substantial zero-shot gains over ReAct and similar prompting paradigms on frontier proprietary models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 35 references that are in the map. First posted on arXiv 24 Sep 2026.

Builds on

28 of 35 references.

References · 28

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  5. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  6. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  7. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
  8. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
  9. 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
  10. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 467
  11. 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agentcited by 246
  12. 2025BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinesecited by 145
16 more
  1. 2025ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarizationcited by 141
  2. 2025WebShaper: Agentically Data Synthesizing via Information-Seeking Formalizationcited by 121
  3. 2025AgentFold: Long-Horizon Web Agents with Proactive Context Managementcited by 103
  4. 2025WebExplorer: Explore and Evolve for Training Long-Horizon Web Agentscited by 92
  5. 2025MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scalingcited by 81
  6. 2025Tongyi DeepResearch Technical Reportcited by 66
  7. 2025WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learningcited by 61
  8. 2025DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RLcited by 55
  9. 2026REDSearcher: A Scalable and Cost-Efficient Framework for Long-Horizon Search Agentscited by 49
  10. 2025WebWeaver: Structuring Web-Scale Evidence with Dynamic Outlines for Open-Ended Deep Researchcited by 46
  11. 2026OpenSeeker: Democratizing Frontier Search Agents by Fully Open-Sourcing Training Datacited by 39
  12. 2025IterResearch: Rethinking Long-Horizon Agents with Interaction Scalingcited by 23
  13. 2026OffSeeker: Online Reinforcement Learning Is Not All You Need for Deep Research Agentscited by 12
  14. 2026AgentCPM-Explore: Realizing Long-Horizon Deep Exploration for Edge-Scale Agentscited by 12
  15. 2025InfoFlow: Reinforcing Search Agent Via Reward Density Optimizationcited by 11
  16. 2025InfoAgent: Advancing Autonomous Information-Seeking Agentscited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Search More, Think Less: Rethinking Long-Horizon Agentic Search for Efficiency and Generalization
  2. 2026Argus: Evidence Assembly for Scalable Deep Research Agents
  3. 2026Beyond Outcome Rewards: Step-Level Self-Distilled Policy Optimization for Deep Search Agents
  4. 2026SearchArt: Training Long-Horizon Search Agent with Scalable Synthetic and Verified Task
  5. 2026Chaining the Evidence: Robust Reinforcement Learning for Deep Search Agents with Citation-Aware Rubric Rewards
  6. 2025HierSearch: A Hierarchical Enterprise Deep Search Framework Integrating Local and Web Searches
  7. 2025Fathom-DeepResearch: Unlocking Long Horizon Information Retrieval and Synthesis for SLMs
  8. 2026AREX: Towards a Recursively Self-Improving Agent for Deep Research