mlsanityData to 14 Oct 2026

Divide and Cooperate: Role-Decomposed Multi-Agent LLM Training with Cross-Agent Learning Signals

arXiv 2606.10684 ↗First posted 9 Jun 2026cs.LG cs.AIcited by 2

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Agentic search systems must coordinate evidence acquisition and response generation, yet existing approaches either couple both roles under a single agent objective or decompose them without disentangling their respective contributions to the final outcome. We introduce DAC (Divide and Cooperate), a role-decomposed training framework that, given task-specific external verification signals, trains a searcher and a generator with role-specific cross-verification rewards. DAC allows the generator to abstain when the retrieved evidence appears insufficient, and uses this decision together with externally evaluated search sufficiency to assign appropriate credit to each role. To prevent degenerate over-abstention, we further introduce hard-positive evidence augmentation, which discourages abstaining on sufficient but hard evidence. Across seven general and multi-hop QA benchmarks and two model backbones, DAC consistently outperforms strong single-agent and multi-agent baselines. Controlled evaluations show that these gains arise from coordinated improvements in both search and generation. Our results highlight the importance of explicitly and properly assigning credit across interacting roles when training agentic search systems.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 38 of its 45 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

38 of 45 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026OASES: Outcome-Aligned Search-Evaluation Co-Training for Agentic Search4 Apr
  2. 2026Beyond Monolithic Architectures: A Multi-Agent Search and Knowledge Optimization Framework for Agentic Search8 Jan
  3. 2025An Empirical Study on Reinforcement Learning for Reasoning-Search Interleaved LLM Agents21 May
  4. 2025s3: You Don't Need That Much Data to Train a Search Agent via RL20 May
  5. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Models9 Jan

Other references · 33

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  10. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  11. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  12. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
21 more
  1. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  2. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  3. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  4. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  5. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  6. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
  7. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 336
  8. 2025ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learningcited by 161
  9. 2025Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learningcited by 161
  10. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learningcited by 86
  11. 2024Prover-Verifier Games improve legibility of LLM outputscited by 85
  12. 2024MALT: Improving Reasoning with Multi-Agent LLM Trainingcited by 76
  13. 2024LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answeringcited by 49
  14. 2025Information Gain-based Policy Optimization: A Simple and Effective Approach for Multi-Turn Search Agentscited by 48
  15. 2021Learning to Give Checkable Answers with Prover-Verifier Gamescited by 32
  16. 2025SFR-DeepResearch: Towards Effective Reinforcement Learning for Autonomously Reasoning Single Agentscited by 31
  17. 2026TIPS: Turn-Level Information-Potential Reward Shaping for Search-Augmented LLMscited by 24
  18. 2026Reasoning and Tool-use Compete in Agentic RL:From Quantifying Interference to Disentangled Tuningcited by 17
  19. 2025RLAC: Reinforcement Learning with Adversarial Critic for Free-Form Generation Taskscited by 13
  20. 2025GRAIT: Gradient-Driven Refusal-Aware Instruction Tuning for Effective Hallucination Mitigationcited by 11
  21. 2024Utilize the Flow before Stepping into the Same River Twice: Certainty Represented Knowledge Flow for Refusal-Aware Instruction Tuningcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Knowledge-Aware Iterative Retrieval for Multi-Agent Systems
  2. 2025Do LLM Agents Know How to Ground, Recover, and Assess? A Benchmark for Epistemic Competence in Information-Seeking Agents
  3. 2026RICE-PO: Turning Retrieval Interactions into Credit Signals for Reasoning Agents
  4. 2026Search-R2: Enhancing Search-Integrated Reasoning via Actor-Refiner Collaboration
  5. 2026Agentic-R: Learning to Retrieve for Agentic Search
  6. 2026Collaborative Multi-Agent Test-Time Reinforcement Learning for Reasoning
  7. 2025Beyond Outcome Reward: Decoupling Search and Answering Improves LLM Agents
  8. 2026CoSearch: Joint Training of Reasoning and Document Ranking via Reinforcement Learning for Agentic Search