mlsanityData to 14 Oct 2026

CooperBench: Why Coding Agents Cannot be Your Teammates Yet

arXiv 2601.13295 ↗First posted 19 Jan 2026cs.LG cs.AI cs.CLcited by 23

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Resolving team conflicts requires not only task-specific competence, but also social intelligence to find common ground and build consensus. As AI agents increasingly collaborate on complex work, they must develop coordination capabilities to function as effective teammates. Yet we hypothesize that current agents lack these capabilities. To test this, we introduce CooperBench, a benchmark of over 600 collaborative coding tasks across 12 libraries in 4 programming languages. Each task assigns two agents different features that can be implemented independently but may conflict without proper coordination. Tasks are grounded in real open-source repositories with expert-written tests. Evaluating state-of-the-art coding agents, we observe the curse of coordination: agents achieve on average 30% lower success rates when working together compared to performing both tasks individually. This contrasts sharply with human teams, where adding teammates typically improves productivity. Our analysis reveals three key issues: (1) communication channels become jammed with vague, ill-timed, and inaccurate messages; (2) even with effective communication, agents deviate from their commitments; and (3) agents often hold incorrect expectations about others' plans and communication. Through large-scale simulation, we also observe rare but interesting emergent coordination behavior including role division, resource division, and negotiation. Our research presents a novel benchmark for collaborative coding and calls for a shift from pursuing individual agent capability to developing social intelligence.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 33 of its 52 references that are in the map. First posted on arXiv 19 Jan 2026.

Builds on

33 of 52 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025Tool-RoCo: An Agent-as-Tool Self-organization Large Language Model Benchmark in Multi-robot Cooperation26 Nov
  2. 2025Collab-Overcooked: Benchmarking and Evaluating Large Language Models as Collaborative Agents27 Feb
  3. 2024Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaboration20 Dec
  4. 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks7 Nov
  5. 2024SOTOPIA-π: Interactive Learning of Socially Intelligent Language Agents13 Mar
  6. 2024Language Agents as Optimizable Graphs26 Feb
  7. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents18 Oct
  8. 2023Theory of Mind for Multi-Agent Collaboration via Large Language Models16 Oct
  9. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug
  10. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society31 Mar

Other references · 23

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  3. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  4. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  5. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
  6. 2023ChatDev: Communicative Agents for Software Developmentcited by 919
  7. 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
  8. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 431
  9. 2025Accelerating scientific discovery with Co-Scientistcited by 401
  10. 2018Machine Theory of Mindcited by 226
  11. 2024MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolutioncited by 134
  12. 2024ToMBench: Benchmarking Theory of Mind in Large Language Modelscited by 72
11 more
  1. 2025A Systematic Survey of Automatic Prompt Optimization Techniquescited by 63
  2. 2024AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodologycited by 46
  3. 2021Few-shot Language Coordination by Modeling Theory of Mindcited by 32
  4. 2025SRT-H: A Hierarchical Framework for Autonomous Surgery via Language Conditioned Imitation Learningcited by 26
  5. 2025Sotopia-RL: Reward Design for Social Intelligencecited by 24
  6. 2025WebGuard: Building a Generalizable Guardrail for Web Agentscited by 21
  7. 2025SoMi-ToM: Evaluating Multi-Perspective Theory of Mind in Embodied Social Interactionscited by 16
  8. 2025TOM-SWE: User Mental Modeling For Software Engineering Agentscited by 13
  9. 2025SyncMind: Measuring Agent Out-of-Sync Recovery in Collaborative Software Engineeringcited by 8
  10. 2025HAWK: A Hierarchical Workflow Framework for Multi-Agent Collaborationcited by 4
  11. 2025High entropy leads to symmetry-equivariant policies in Dec-POMDPscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 23.

23 papers.

Sort:

By area:

  1. 2026Effective Strategies for Asynchronous Software Engineering Agentscited by 16
  2. 2026Shepherd: Enabling Programmable Meta-Agents via Reversible Agentic Execution Tracescited by 12
  3. 2026Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failurescited by 10
  4. 2026MT-PingEval: Evaluating Multi-Turn Collaboration with Private Information Gamescited by 8
  5. 2026CalBench: Evaluating Coordination-Privacy Trade-offs in Multi-Agent LLMscited by 7
  6. 2026One Frozen Simulator Is Not Enough: Simulator Collapse in Multi-Agent RLcited by 6
  7. 2026When Parallelism Pays Off: Cohesion-Aware Task Partitioning for Multi-Agent Codingcited by 5
  8. 2026El Agente Gr'afico: A Semantic Execution Runtime for Scientific Agentscited by 4
  9. 2026The Topological Trouble With Transformerscited by 3
  10. 2026Interactive Evaluation Requires a Design Sciencecited by 3
  11. 2026Emergent Collusion in Long-Horizon LLM Agent Interactioncited by 2
  12. 2026Talk is Cheap, Communication is Hard: Dynamic Grounding Failures and Repair in Multi-Agent Negotiationcited by 1
11 more
  1. 2026Claim Plane: Enforceable Change Intents and Dynamic Scope for Parallel Coding Agentscited by 1
  2. 2026GlossoGen: Emergent Language in Complex Multi-Agent LLM Interactionscited by 1
  3. 2026Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluationcited by 1
  4. 2026Beyond Code Generation: Reliability, Verification, and Cost Economics in the Agentic Software Development Lifecyclecited by 1
  5. 2026OpenCollab: A Multi-Agent Coding Framework with Programmable Collaboration and Controllable Runtimecited by 1
  6. 2026Multi-agent AI systems outperform human teams in creativitycited by 0
  7. 2026Claim Plane: Reliability Gains and the Limits of Selective Concurrency for Parallel Coding Agents: A 30-Pair, Three-Seed Confirmatory Study of Deterministic Pre-Write Admissioncited by 0
  8. 2026AsynCodeBench: Benchmarking Collaboration of Asynchronous Multi-Agent Systems in Software Engineeringcited by 0
  9. 2026Relic: From Multi-Agent Collaboration to Persistent Organizational Capabilitycited by 0
  10. 2026Scaling Inference-Time Compute with Teams of Coding Agentscited by 0
  11. 2026Choosing LLMs for Strategic Agent Interactionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026When Agents Coordinate: Measuring Coordination in Multi-Agent AI Coding
  2. 2026Humans are Missing from AI Coding Agent Research
  3. 2025The Collaboration Gap: Exploration and Benchmarking of Open-World Agentic Cooperation
  4. 2023Cooperation on the Fly: Exploring Language Agents for Ad Hoc Teamwork in the Avalon Game
  5. 2026CollabSim: A CSCW-Grounded Methodology for Investigating Collaborative Competence of LLM Agents through Controlled Multi-Agent Experiments
  6. 2026COOP2: Defining, Observing, and Repairing Cooperation in LLM Multi-Agent Systems
  7. 2026More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaboration
  8. 2026Beyond Perfect Teammates: Multi-Agent Collaboration with Unreliable Colleagues