mlsanityData to 7 Oct 2026

Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agents

arXiv 2402.12327 ↗First posted 19 Feb 2024cs.AI cs.CL cs.CYcited by 48

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

Large Language Models (LLMs) have increasingly been utilized in social simulations, where they are often guided by carefully crafted instructions to stably exhibit human-like behaviors during simulations. Nevertheless, we doubt the necessity of shaping agents' behaviors for accurate social simulations. Instead, this paper emphasizes the importance of spontaneous phenomena, wherein agents deeply engage in contexts and make adaptive decisions without explicit directions. We explored spontaneous cooperation across three competitive scenarios and successfully simulated the gradual emergence of cooperation, findings that align closely with human behavioral data. This approach not only aids the computational social science community in bridging the gap between simulations and real-world dynamics but also offers the AI community a novel method to assess LLMs' capability of deliberate reasoning.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 33 of its 77 references that are in the map. First posted on arXiv 19 Feb 2024.

Builds on

33 of 77 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023LLM-Based Agent Society Investigation: Collaboration and Confrontation in Avalon Gameplay23 Oct
  2. 2023Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View3 Oct
  3. 2023Exploring the Intersection of Large Language Models and Agent-Based Modeling via Prompt Engineering14 Aug
  4. 2023Generative Agents: Interactive Simulacra of Human Behavior7 Apr

Other references · 29

  1. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  2. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  3. 2023A Survey of Large Language Modelscited by 2,537
  4. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  5. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  6. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  7. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  8. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
  9. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
  10. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 506
  11. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 253
  12. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 238
17 more
  1. 2023Evaluating Large Language Models in Theory of Mind Taskscited by 217
  2. 2024Quantifying the Persona Effect in LLM Simulationscited by 164
  3. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 161
  4. 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 154
  5. 2023OpenAgents: An Open Platform for Language Agents in the Wildcited by 136
  6. 2023Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiationcited by 94
  7. 2024Revolutionizing Finance with LLMs: An Overview of Applications and Insightscited by 88
  8. 2023AgentSims: An Open-Source Sandbox for Large Language Model Evaluationcited by 86
  9. 2023BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agentscited by 83
  10. 2024Enhance Reasoning for Large Language Models in the Game Werewolfcited by 44
  11. 2023Cooperation on the Fly: Exploring Language Agents for Ad Hoc Teamwork in the Avalon Gamecited by 27
  12. 2018Learning to Collaborate: Multi-Scenario Ranking via Multi-Agent Reinforcement Learningcited by 23
  13. 2023GPT-Based Models Meet Simulation: How to Efficiently Use Large-Scale Pre-Trained Language Models Across Simulation Taskscited by 14
  14. 2023"Guinea Pig Trials" Utilizing GPT: A Novel Smart Agent-Based Modeling Approach for Studying Firm Competition and Collusioncited by 13
  15. 2023Smart Agent-Based Modeling: On the Use of Large Language Models in Computer Simulationscited by 11
  16. 2024CCA: Collaborative Competitive Agents for Image Editingcited by 9
  17. 2020Learning to Collaborate in Multi-Module Recommendation via Multi-Agent Reinforcement Learning without Communicationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 23; 2026: 21.

48 papers.

Sort:

By area:

  1. 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Modelscited by 97
  2. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 76
  3. 2024MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debatecited by 54
  4. 2025Do Large Language Models Solve the Problems of Agent-Based Modeling? A Critical Review of Generative Social Simulationscited by 28
  5. 2025Reinforce LLM Reasoning through Multi-Agent Reflectioncited by 26
  6. 2024InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Contextcited by 25
  7. 2025TAMAS: Benchmarking Adversarial Risks in Multi-Agent LLM Systemscited by 24
  8. 2025Towards a Design Guideline for RPA Evaluation: A Survey of Large Language Model-Based Role-Playing Agentscited by 22
  9. 2025The Coming Crisis of Multi-Agent Misalignment: AI Alignment Must Be a Dynamic and Social Processcited by 21
  10. 2026Lying with Truths: Open-Channel Multi-Agent Collusion for Belief Manipulation via Generative Montagecited by 17
  11. 2026Colosseum: Auditing Collusion in Cooperative Multi-Agent Systemscited by 17
  12. 2026Does Socialization Emerge in AI Agent Society? A Case Study of Moltbookcited by 13
36 more
  1. 2025Seven Security Challenges in Cross-domain Multi-agent LLM Systemscited by 12
  2. 2025The PIMMUR Principles: Ensuring Validity in Collective Behavior of LLM Societiescited by 12
  3. 2025The Hidden Strength of Disagreement: Unraveling the Consensus-Diversity Tradeoff in Adaptive Multi-Agent Systemscited by 11
  4. 2025LLM-Based Social Simulations Require a Boundarycited by 11
  5. 2025Evaluating LLM Agent Collusion in Double Auctionscited by 11
  6. 2025Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Surveycited by 10
  7. 2025Spontaneous Giving and Calculated Greed in Language Modelscited by 9
  8. 2025Many LLMs Are More Utilitarian Than Onecited by 7
  9. 2025Collab-REC: An LLM-based Agentic Framework for Balancing Recommendations in Tourismcited by 7
  10. 2025Stop Reducing Responsibility in LLM-Powered Multi-Agent Systems to Local Alignmentcited by 7
  11. 2026The Devil Behind Moltbook: Anthropic Safety is Always Vanishing in Self-Evolving AI Societiescited by 6
  12. 2026Evaluating Collective Behaviour of Hundreds of LLM Agentscited by 6
  13. 2025AI Agent Behavioral Sciencecited by 5
  14. 2026RADAR: Redundancy-Aware Diffusion for Multi-Agent Communication Structure Generationcited by 5
  15. 2025Large Language Models for Agent-Based Modelling: Current and possible uses across the modelling cyclecited by 4
  16. 2025The Other Mind: How Language Models Exhibit Human Temporal Cognitioncited by 4
  17. 2026Human Values Matter: Investigating How Misalignment Shapes Collective Behaviors in LLM Agent Communitiescited by 4
  18. 2026Social Theory Should Be a Structural Prior for Agentic AI: A Formal Framework for Multi-Agent Social Systemscited by 4
  19. 2025Can Competition Enhance the Proficiency of Agents Powered by Large Language Models in the Realm of News-driven Time Series Forecasting?cited by 2
  20. 2025Communication and Verification in LLM Agents towards Collaboration under Information Asymmetrycited by 2
  21. 2026Federation over Text: Insight Sharing for Multi-Agent Reasoningcited by 2
  22. 2026If LLMs Have Human-Like Attributes, Then So Does Age of Empires IIcited by 2
  23. 2025Shall We Play a Game? Language Models for Open-ended Wargamescited by 1
  24. 2026Harm in AI-Driven Societies: An Audit of Toxicity Adoption on Chirper.aicited by 1
  25. 2026GameTalk: Training LLMs for Strategic Conversationcited by 1
  26. 2026Interaction Theater: A case of LLM Agents Interacting at Scalecited by 1
  27. 2026Superminds Test: Actively Evaluating Collective Intelligence of Agent Society via Probing Agentscited by 1
  28. 2025A Dynamic and High-Precision Method for Scenario-Based HRA Synthetic Data Collection in Multi-Agent Collaborative Environments Driven by LLMscited by 0
  29. 2025SRA-CP: Spontaneous Risk-Aware Selective Cooperative Perceptioncited by 0
  30. 2026Supracompetitive Pricing Under AI Monoculturecited by 0
  31. 2026Aligned but Not Partner-Specific: Distinguishing How Multimodal LLM Agents Succeed in Reference Games Without Human-Like Conventionscited by 0
  32. 2026Is Lying an Emergent Behaviour in LLMs? Evidence from Gaslighting AI agents in a Sustainability Gamecited by 0
  33. 2026How Affect Propagates among LLM Agents: Emergent Emotional Contagion in Crowd Simulationcited by 0
  34. 2026Eco3S: Complex Socio-Economic System Simulation via Agent-Based Modelscited by 0
  35. 2026Scaling Multi-Agent Systems with Prospect-State Propagationcited by 0
  36. 2026Population Physics, Population Problems: Safety and Emergence in LLM Societiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Evolution of Cooperation in LLM-Agent Societies: A Preliminary Study Using Different Punishment Strategies
  2. 2026Collective cooperation without individual fidelity in LLM agents
  3. 2026AI Agents Alone Are Not (Yet) Sufficient for Social Simulation
  4. 2024Static network structure cannot stabilize cooperation among Large Language Model agents
  5. 2025Will Systems of LLM Agents Cooperate: An Investigation into a Social Dilemma
  6. 2025Emergent Social Dynamics of LLM Agents in the El Farol Bar Problem
  7. 2026SODE: Analyzing Social Dynamics in LLM Agents
  8. 2024LLM-Augmented Agent-Based Modelling for Social Simulations: Challenges and Opportunities