mlsanityData to 7 Oct 2026

The Subtle Art of Defection: Understanding Uncooperative Behaviors in LLM based Multi-Agent Systems

arXiv 2511.15862 ↗First posted 19 Nov 2025cs.MA cs.CLcited by 3

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

This paper introduces a novel framework for simulating and analyzing how uncooperative behaviors can destabilize or collapse LLM-based multi-agent systems. Our framework includes two key components: (1) a game theory-based taxonomy of uncooperative agent behaviors, addressing a notable gap in the existing literature; and (2) a structured, multi-stage simulation pipeline that dynamically generates and refines uncooperative behaviors as agents' states evolve. We evaluate the framework via a collaborative resource management setting, measuring system stability using metrics such as survival time and resource overuse rate. Empirically, our framework achieves 96.7% accuracy in generating realistic uncooperative behaviors, validated by human evaluations. Our results reveal a striking contrast: cooperative agents maintain perfect system stability (100% survival over 12 rounds with 0% resource overuse), while any uncooperative behavior can trigger rapid system collapse within 1 to 7 rounds. We also evaluate LLM-based defense methods, finding they detect some uncooperative behaviors, but some behaviors remain largely undetectable. These gaps highlight how uncooperative agents degrade collective outcomes and underscore the need for more resilient multi-agent systems.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 8 of its 18 references that are in the map. First posted on arXiv 19 Nov 2025.

Builds on

8 of 18 references.

References · 8

  1. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 406
  2. 2023EconAgent: Large Language Model-Empowered Agents for Simulating Macroeconomic Activitiescited by 154
  3. 2024Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agentscited by 120
  4. 2025Red-Teaming LLM Multi-Agent Systems via Communication Attackscited by 118
  5. 2024PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safetycited by 79
  6. 2024Trust No Bot: Discovering Personal Disclosures in Human-LLM Conversations in the Wildcited by 43
  7. 2025GUARDIAN: Safeguarding LLM Multi-Agent Collaborations with Temporal Graph Modelingcited by 35
  8. 2025Emergent Crowds Dynamics from Language-Driven Multi-Agent Interactionscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systemscited by 1
  2. 2026From Certain Doom to Survival: Agent-Driven Self-Governance in LLM Agent Societiescited by 0
  3. 2026Population Physics, Population Problems: Safety and Emergence in LLM Societiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Static Sandboxes Are Inadequate: Modeling Societal Complexity Requires Open-Ended Co-Evolution in LLM-Based Multi-Agent Simulations
  2. 2025Risk Analysis Techniques for Governed LLM-based Multi-Agent Systems
  3. 2026Cooperation Breakdown in LLM Agents Under Communication Delays
  4. 2026Silent Failure in LLM Agent Systems: The Entropy Principle and the Inevitable Disorder of Autonomous Agents
  5. 2026MAS-FIRE: Fault Injection and Reliability Evaluation for LLM-Based Multi-Agent Systems
  6. 2025Will Systems of LLM Agents Cooperate: An Investigation into a Social Dilemma
  7. 2026From Task Solving to Robust Real-World Adaptation in LLM Agents
  8. 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systems