mlsanityData to 7 Oct 2026

More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaboration

arXiv 2604.07821 ↗First posted 9 Apr 2026cs.MA cs.AI cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Large language model (LLM) agents increasingly coordinate in multi-agent systems, yet we lack an understanding of where and why cooperation fails. Many real-world coordination problems are not social dilemmas: helping others -- sharing documentation, unblocking a teammate -- costs the helper almost nothing while producing substantial collective benefit. Whether LLM agents cooperate in this regime, where helping is free and they are explicitly instructed to do so, remains unknown. We build a turn-based multi-agent environment that strips away all strategic complexity, making cooperation costless and trivially optimal. Across eight widely used LLMs, capability does not predict cooperation: OpenAI o3 reaches only 17% of optimal collective performance while the weaker o3-mini reaches 50%, despite identical instructions to maximize group revenue. Using a causal decomposition that automates one side of agent communication, we separate cooperation failures from competence failures, and find that several capable models actively withhold information despite gaining nothing from withholding. Targeted interventions address each mode: explicit protocols roughly double the performance of competence-limited models, while small sharing incentives unlock cooperation-limited ones. Our results suggest that scaling intelligence alone will not solve coordination in multi-agent systems, and will require deliberate cooperative design, even when helping costs nothing.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 17 of its 60 references that are in the map. First posted on arXiv 9 Apr 2026.

Builds on

17 of 60 references.

References · 17

  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  2. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  3. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  4. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  5. 2016Learning Multiagent Communication with Backpropagationcited by 355
  6. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 319
  7. 2025AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Societycited by 180
  8. 2024Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agentscited by 120
  9. 2019On the Pitfalls of Measuring Emergent Communicationcited by 66
  10. 2019Learning Efficient Multi-agent Communication: An Information Bottleneck Approachcited by 60
  11. 2024Cultural Evolution of Cooperation among LLM Agentscited by 30
  12. 2025Will Systems of LLM Agents Cooperate: An Investigation into a Social Dilemmacited by 22
5 more
  1. 2025Corrupted by Reasoning: Reasoning Language Models Become Free-Riders in Public Goods Gamescited by 20
  2. 2021Adaptive Incentive Design with Multi-Agent Meta-Gradient Reinforcement Learningcited by 18
  3. 2023Cheap Talk Discovery and Utilization in Multi-Agent Reinforcement Learningcited by 4
  4. 2024Observation Interference in Partially Observable Assistance Gamescited by 4
  5. 2025Humans expect rationality and cooperation from LLM opponents in strategic gamescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026COOP2: Defining, Observing, and Repairing Cooperation in LLM Multi-Agent Systems
  2. 2026Payoff scaling shapes cooperation in LLM agents across languages
  3. 2025Collaborate, Deliberate, Evaluate: How LLM Alignment Affects Coordinated Multi-Agent Outcomes
  4. 2024Embodied LLM Agents Learn to Cooperate in Organized Teams
  5. 2026Collective cooperation without individual fidelity in LLM agents
  6. 2025Communication and Verification in LLM Agents towards Collaboration under Information Asymmetry
  7. 2025Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs
  8. 2025Multi-Agent Language Models: Advancing Cooperation, Coordination, and Adaptation