mlsanityData to 14 Oct 2026

Evolution of Cooperation in LLM-Agent Societies: A Preliminary Study Using Different Punishment Strategies

arXiv 2504.19487 ↗First posted 28 Apr 2025cs.MAcited by 6

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

The evolution of cooperation has been extensively studied using abstract mathematical models and simulations. Recent advances in Large Language Models (LLMs) and the rise of LLM agents have demonstrated their ability to perform social reasoning, thus providing an opportunity to test the emergence of norms in more realistic agent-based simulations with human-like reasoning using natural language. In this research, we investigate whether the cooperation dynamics presented in Boyd and Richerson's model persist in a more realistic simulation of the Diner's Dilemma using LLM agents compared to the abstract mathematical nature in the work of Boyd and Richerson. Our findings indicate that agents follow the strategies defined in the Boyd and Richerson model, and explicit punishment mechanisms drive norm emergence, reinforcing cooperative behaviour even when the agent strategy configuration varies. Our results suggest that LLM-based Multi-Agent System simulations, in fact, can replicate the evolution of cooperation predicted by the traditional mathematical models. Moreover, our simulations extend beyond the mathematical models by integrating natural language-driven reasoning and a pairwise imitation method for strategy adoption, making them a more realistic testbed for cooperative behaviour in MASs.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 19 of its 42 references that are in the map. First posted on arXiv 28 Apr 2025.

Builds on

19 of 42 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Harnessing the power of LLMs for normative reasoning in MASs25 Mar
  2. 2024Norm Violation Detection in Multi-Agent Systems using Large Language Models: A Pilot Study25 Mar
  3. 2024Emergence of Social Norms in Generative Agent Societies: Principles and Architecture13 Mar
  4. 2023Do LLM Agents Exhibit Social Behavior?23 Dec
  5. 2023ALYMPICS: LLM Agents Meet Game Theory -- Exploring Strategic Decision-Making with AI Agents6 Nov
  6. 2023Generative Agents: Interactive Simulacra of Human Behavior7 Apr

Other references · 13

  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  2. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  3. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 598
  4. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 327
  5. 2023Playing repeated games with Large Language Modelscited by 217
  6. 2020Watch-And-Help: A Challenge for Social Perception and Human-AI Collaborationcited by 133
  7. 2023Can Large Language Models Serve as Rational Players in Game Theory? A Systematic Analysiscited by 92
  8. 2024Nicer Than Humans: How do Large Language Models Behave in the Prisoner's Dilemma?cited by 63
  9. 2024Can Language Models Serve as Text-Based World Simulators?cited by 52
  10. 2022Melting Pot 2.0cited by 40
  11. 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approachescited by 18
  12. 2024Extracting Norms from Contracts Via ChatGPT: Opportunities and Challengescited by 2
1 more
  1. 2024Cooperative bots exhibit nuanced effects on cooperation across strategic frameworkscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Will Systems of LLM Agents Cooperate: An Investigation into a Social Dilemma
  2. 2024Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agents
  3. 2024Evolution of Social Norms in LLM Agents using Natural Language
  4. 2026Collective cooperation without individual fidelity in LLM agents
  5. 2026Payoff scaling shapes cooperation in LLM agents across languages
  6. 2024Cultural Evolution of Cooperation among LLM Agents
  7. 2024Static network structure cannot stabilize cooperation among Large Language Model agents
  8. 2024Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents