mlsanityData to 14 Oct 2026

Training LLMs for Multi-Agent Collaboration with Verifiable Information Rewards

arXiv 2610.17893 ↗First posted 13 Oct 2026cs.AI cs.CL cs.LGcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents also fits: General LLM agent frameworks and benchmarks

Abstract (arXiv)

Effective collaboration among large language model (LLM) agents in multi-agent systems requires sharing relevant evidence, requesting missing information, and incorporating teammates' contributions into decisions. Training these behaviors is challenging because final task outcomes provide limited feedback on the value of individual messages. We introduce CoSIR (Collaboration via Surprisal-based Information Rewards), a post-training approach that uses a simple, verifiable environment to teach collaboration behaviors that generalize to more complex, unseen tasks. We propose a Bayesian evaluator that scores shared information by its reduction in team surprisal, defined as the sum of the negative log probabilities assigned to the correct option given each agent's available evidence. Requests are scored by the potential reduction in the same quantity if the requested information were truthfully provided. Separate rewards encourage agents to use the received information in their private choices and calculations. We post-train Qwen3-4B, Qwen3-8B, and Gemma-3-4B-it through self-play and evaluate transfer to seven unseen natural-language collaboration tasks. CoSIR achieves the highest aggregate score, while its performance in general tasks remains competitive. Behavioral analyses show increased disclosure of private evidence and fewer judged failures to use teammates' input.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 19 of its 28 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

19 of 28 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026Learning Decentralized LLM Collaboration with Multi-Agent Actor Critic29 Jan
  2. 2025SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning30 Jun
  3. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning24 Apr
  4. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learning25 Feb

Other references · 15

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
  7. 20218-bit Optimizers via Block-wise Quantizationcited by 403
  8. 2025SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Modelcited by 335
  9. 2019Mastering Complex Control in MOBA Games with Deep Reinforcement Learningcited by 161
  10. 2023MathChat: Converse to Tackle Challenging Math Problems with LLM Agentscited by 68
  11. 2025Do as We Do, Not as You Think: the Conformity of Large Language Modelscited by 53
  12. 2026Silo-Bench: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systemscited by 20
3 more
  1. 2025Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMscited by 19
  2. 2026HIPO: Instruction Hierarchy via Constrained Reinforcement Learningcited by 3
  3. 2026The Collaboration Tax: How Much LLM Multi-Agent Systems Pay to Coordinatecited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Collaborative Belief Reasoning with LLMs for Efficient Multi-Agent Collaboration
  2. 2025Communication and Verification in LLM Agents towards Collaboration under Information Asymmetry
  3. 2025Reinforcement Learning-Augmented LLM Agents for Collaborative Decision Making and Performance Optimization
  4. 2024ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaboration
  5. 2024Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration
  6. 2026More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaboration
  7. 2026Improving LLM Collaboration via Multi-Agent Preference Learning
  8. 2025Collab-Overcooked: Benchmarking and Evaluating Large Language Models as Collaborative Agents