mlsanityData to 14 Oct 2026

Cultural Evolution of Cooperation among LLM Agents

arXiv 2412.10270 ↗First posted 13 Dec 2024cs.MA cs.AIcited by 32

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

Large language models (LLMs) provide a compelling foundation for building generally-capable AI agents. These agents may soon be deployed at scale in the real world, representing the interests of individual humans (e.g., AI assistants) or groups of humans (e.g., AI-accelerated corporations). At present, relatively little is known about the dynamics of multiple LLM agents interacting over many generations of iterative deployment. In this paper, we examine whether a "society" of LLM agents can learn mutually beneficial social norms in the face of incentives to defect, a distinctive feature of human sociality that is arguably crucial to the success of civilization. In particular, we study the evolution of indirect reciprocity across generations of LLM agents playing a classic iterated Donor Game in which agents can observe the recent behavior of their peers. We find that the evolution of cooperation differs markedly across base models, with societies of Claude 3.5 Sonnet agents achieving significantly higher average scores than Gemini 1.5 Flash, which, in turn, outperforms GPT-4o. Further, Claude 3.5 Sonnet can make use of an additional mechanism for costly punishment to achieve yet higher scores, while Gemini 1.5 Flash and GPT-4o fail to do so. For each model class, we also observe variation in emergent behavior across random seeds, suggesting an understudied sensitive dependence on initial conditions. We suggest that our evaluation regime could inspire an inexpensive and informative new class of LLM benchmarks, focussed on the implications of LLM agent deployment for the cooperative infrastructure of society.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 18 of its 44 references that are in the map. First posted on arXiv 13 Dec 2024.

Builds on

18 of 44 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Artificial Leviathan: Exploring Social Evolution of LLM Agents Through the Lens of Hobbesian Social Contract Theory20 Jun
  2. 2023Do LLM Agents Exhibit Social Behavior?23 Dec
  3. 2023Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia6 Dec
  4. 2023CompeteAI: Understanding the Competition Dynamics in Large Language Model-based Agents26 Oct
  5. 2023Generative Agents: Interactive Simulacra of Human Behavior7 Apr

Other references · 13

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  3. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  4. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 551
  5. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
  6. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 241
  7. 2023Playing repeated games with Large Language Modelscited by 217
  8. 2024The Ethics of Advanced AI Assistantscited by 107
  9. 2023Strategic Reasoning with Language Modelscited by 64
  10. 2023Machine Culturecited by 55
  11. 2023Social Learning: Towards Collaborative Learning with Large Language Modelscited by 15
  12. 2024Cultural evolution in populations of Large Language Modelscited by 13
1 more
  1. 2024Collective Innovation in Groups of Large Language Modelscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 21.

32 papers.

Sort:

By area:

  1. 2025Agentic Large Language Models, a surveycited by 108
  2. 2025Corrupted by Reasoning: Reasoning Language Models Become Free-Riders in Public Goods Gamescited by 20
  3. 2025Spontaneous Giving and Calculated Greed in Language Modelscited by 10
  4. 2025How Personality Traits Shape LLM Risk-Taking Behaviourcited by 9
  5. 2025The Role of Social Learning and Collective Norm Formation in Fostering Cooperation in LLM Multi-Agent Systemscited by 9
  6. 2025AstroAgents: A Multi-Agent AI for Hypothesis Generation from Mass Spectrometry Datacited by 8
  7. 2025Modeling Earth-Scale Human-Like Societies with One Billion Agentscited by 8
  8. 2025Moloch's Bargain: Emergent Misalignment When LLMs Compete for Audiencescited by 8
  9. 2026Evaluating Collective Behaviour of Hundreds of LLM Agentscited by 8
  10. 2026OpenClaw AI Agents as Informal Learners at Moltbook: Characterizing an Emergent Learning Community at Scalecited by 8
  11. 2026More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaborationcited by 7
  12. 2025Emergent Coordinated Behaviors in Networked LLM Agents: Modeling the Strategic Dynamics of Information Operationscited by 6
20 more
  1. 2026Gender Dynamics and Homophily in a Social Network of LLM Agentscited by 5
  2. 2026Talk, Judge, Cooperate: Gossip-Driven Indirect Reciprocity in Self-Interested LLM Agentscited by 5
  3. 2026When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Communitycited by 3
  4. 2025Recursive Training Loops in LLMs: How training data properties modulate distribution shift in generated data?cited by 2
  5. 2025From Connectivity to Autonomy: The Dawn of Self-Evolving Communication Systemscited by 1
  6. 2026Learning to Learn from Language Feedback with Social Meta-Learningcited by 1
  7. 2026When AI Agents Learn from Each Other: Insights from Emergent AI Agent Communities on OpenClaw for Human-AI Partnership in Educationcited by 1
  8. 2026Emergence of Reputation-Based Cooperation in LLM Agentscited by 1
  9. 2026Behavior is Not Enough: A Mechanism-Based Evaluation of Social Norm Emergence in LLM Societiescited by 1
  10. 2026Position: Introspective Experience from Conversational Environments as a Path to Better Learningcited by 0
  11. 2026Next-Token Prediction and Regret Minimizationcited by 0
  12. 2026You Only Align Once: Propagating Cooperative Behaviors in Multi-Agent Systems through Seed Agentscited by 0
  13. 2026Evolutionary Dynamics of Cooperation in Next-Generation LLM Agent Systems: A Cross-Provider Empirical Extensioncited by 0
  14. 2026Emergent Relational Order in LLM Agent Societies: From Collective Affect to Authority Stratificationcited by 0
  15. 2026Not a Monolith: Lab-Level Divergence in the Cooperative Equilibria of Chinese Frontier LLM Agentscited by 0
  16. 2026Patterns of Priming in Production: Lexical, Semantic and Structural Alignment in Language Model Generationcited by 0
  17. 2026Playing social deduction games with reinforcement fine-tuned large language modelscited by 0
  18. 2026Communication Shapes Collective Inference in Self-Adapting LLM Societies: Evidence from Mafiacited by 0
  19. 2026A Society of Researchers: Designing Institutions for Populations of Autonomous Research Agentscited by 0
  20. 2026Why LLM Agents Favor Their Group: Stakes, Observed Norms, and Reputationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Evolution of Cooperation in LLM-Agent Societies: A Preliminary Study Using Different Punishment Strategies
  2. 2026Collective cooperation without individual fidelity in LLM agents
  3. 2026Payoff scaling shapes cooperation in LLM agents across languages
  4. 2026SODE: Analyzing Social Dynamics in LLM Agents
  5. 2026Group Selection Promotes Prosocial Prompts in Populations of LLM Agents
  6. 2024Static network structure cannot stabilize cooperation among Large Language Model agents
  7. 2025Will Systems of LLM Agents Cooperate: An Investigation into a Social Dilemma
  8. 2024Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agents