More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaboration
Not on the map. Why?
Abstract (arXiv)
Large language model (LLM) agents increasingly coordinate in multi-agent systems, yet we lack an understanding of where and why cooperation fails. Many real-world coordination problems are not social dilemmas: helping others -- sharing documentation, unblocking a teammate -- costs the helper almost nothing while producing substantial collective benefit. Whether LLM agents cooperate in this regime, where helping is free and they are explicitly instructed to do so, remains unknown. We build a turn-based multi-agent environment that strips away all strategic complexity, making cooperation costless and trivially optimal. Across eight widely used LLMs, capability does not predict cooperation: OpenAI o3 reaches only 17% of optimal collective performance while the weaker o3-mini reaches 50%, despite identical instructions to maximize group revenue. Using a causal decomposition that automates one side of agent communication, we separate cooperation failures from competence failures, and find that several capable models actively withhold information despite gaining nothing from withholding. Targeted interventions address each mode: explicit protocols roughly double the performance of competence-limited models, while small sharing incentives unlock cooperation-limited ones. Our results suggest that scaling intelligence alone will not solve coordination in multi-agent systems, and will require deliberate cooperative design, even when helping costs nothing.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 17 of its 60 references that are in the map. First posted on arXiv 9 Apr 2026.
Builds on
17 of 60 references.
References · 17
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
- 2016Learning Multiagent Communication with Backpropagationcited by 355
- 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 319
- 2025AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Societycited by 180
- 2024Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agentscited by 120
- 2019On the Pitfalls of Measuring Emergent Communicationcited by 66
- 2019Learning Efficient Multi-agent Communication: An Information Bottleneck Approachcited by 60
- 2024Cultural Evolution of Cooperation among LLM Agentscited by 30
- 2025Will Systems of LLM Agents Cooperate: An Investigation into a Social Dilemmacited by 22
5 more
- 2025Corrupted by Reasoning: Reasoning Language Models Become Free-Riders in Public Goods Gamescited by 20
- 2021Adaptive Incentive Design with Multi-Agent Meta-Gradient Reinforcement Learningcited by 18
- 2023Cheap Talk Discovery and Utilization in Multi-Agent Reinforcement Learningcited by 4
- 2024Observation Interference in Partially Observable Assistance Gamescited by 4
- 2025Humans expect rationality and cooperation from LLM opponents in strategic gamescited by 3
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026CollabSim: A CSCW-Grounded Methodology for Investigating Collaborative Competence of LLM Agents through Controlled Multi-Agent Experimentscited by 2
- 2026Draining the Energy Commons: Self-Defeating Over-Appropriation as a Coordination Failure in Agentic LLM Collectivescited by 1
- 2026The Collaboration Tax: How Much LLM Multi-Agent Systems Pay to Coordinatecited by 1
- 2026ToMAS: A Pilot Failure-Grounded Theory-of-Mind Benchmark from Multi-Agent LLM Failurescited by 0
- 2026From Certain Doom to Survival: Agent-Driven Self-Governance in LLM Agent Societiescited by 0
- 2026The Decomposition Tax: LLM Pipelines Lose Up to 40 Accuracy Points at Their Own Interfacescited by 0
- 2026Multi-agent discussion gains less when dissent is withheldcited by 0
- 2026Multi-agent discussion gains less when dissent is withheld29 Sep 2026
- 2026The Decomposition Tax: LLM Pipelines Lose Up to 40 Accuracy Points at Their Own Interfaces26 Sep 2026
- 2026From Certain Doom to Survival: Agent-Driven Self-Governance in LLM Agent Societies18 Sep 2026
- 2026ToMAS: A Pilot Failure-Grounded Theory-of-Mind Benchmark from Multi-Agent LLM Failures15 Sep 2026
- 2026The Collaboration Tax: How Much LLM Multi-Agent Systems Pay to Coordinate23 Aug 2026
- 2026Draining the Energy Commons: Self-Defeating Over-Appropriation as a Coordination Failure in Agentic LLM Collectives24 Jul 2026
- 2026CollabSim: A CSCW-Grounded Methodology for Investigating Collaborative Competence of LLM Agents through Controlled Multi-Agent Experiments4 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026COOP2: Defining, Observing, and Repairing Cooperation in LLM Multi-Agent Systems
- 2026Payoff scaling shapes cooperation in LLM agents across languages
- 2025Collaborate, Deliberate, Evaluate: How LLM Alignment Affects Coordinated Multi-Agent Outcomes
- 2024Embodied LLM Agents Learn to Cooperate in Organized Teams
- 2026Collective cooperation without individual fidelity in LLM agents
- 2025Communication and Verification in LLM Agents towards Collaboration under Information Asymmetry
- 2025Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs
- 2025Multi-Agent Language Models: Advancing Cooperation, Coordination, and Adaptation