mlsanityData to 7 Oct 2026

From Certain Doom to Survival: Agent-Driven Self-Governance in LLM Agent Societies

arXiv 2609.22600 ↗First posted 18 Sep 2026cs.MAcited by 0

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

Multi-agent LLM systems are increasingly evaluated in social dilemmas, but most work treats governance as imposed by the experimenter, expressed rhetorically, or restricted to a fixed menu of mechanisms. We introduce GovSim-SelfGovern, an extension of the GovSim common-pool resource environment in which agents author executable Python governance rules, receive sandbox validation feedback, vote on proposed laws, and live under the rules they enact across rounds. To evaluate agent-driven self-governance, we examine three scenarios ranging from stable abundance to a fatal resource wall where five agents cannot all survive through harvest alone. To solve this, agents must write and debug useful laws in time before their institutions degrade sharply under resource pressure. Finally, we study a central alignment question: when agents hesitate to propose exile, are they rejecting it for normative reasons, or does it never enter their candidate set? Our results show that executable governance improves the space of possible interventions for agents, but survival depends on whether agents discover the right institutional mechanisms in time. Fiscal capacity enables redistribution, while deeper reasoning and removal of democratic veto make exile more feasible. GovSim-SelfGovern therefore adapts executable code actions to a common-pool governance setting and shows how scarcity turns institutional authorship into a political and ethical problem.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 34 of its 41 references that are in the map. First posted on arXiv 18 Sep 2026.

Builds on

34 of 41 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026CoopEval: Benchmarking Cooperation-Sustaining Mechanisms and LLM Agents in Social Dilemmas16 Apr
  2. 2026Human Values Matter: Investigating How Misalignment Shapes Collective Behaviors in LLM Agent Communities7 Apr
  3. 2026Evaluating Collective Behaviour of Hundreds of LLM Agents18 Feb
  4. 2025The Role of Social Learning and Collective Norm Formation in Fostering Cooperation in LLM Multi-Agent Systems16 Oct
  5. 2024An Electoral Approach to Diversify LLM-based Multi-Agent Collective Decision-Making19 Oct
  6. 2024Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents25 Apr
  7. 2023Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia6 Dec
  8. 2023Generative Agents: Interactive Simulacra of Human Behavior7 Apr

Other references · 26

  1. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  2. 2024Executable Code Actions Elicit Better LLM Agentscited by 537
  3. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 506
  4. 2017Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 206
  5. 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 174
  6. 2025Multi-Agent Risks from Advanced AIcited by 150
  7. 2018Inequity aversion improves cooperation in intertemporal social dilemmascited by 106
  8. 2024Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedbackcited by 95
  9. 2017A multi-agent reinforcement learning model of common-pool resource appropriationcited by 68
  10. 2025Corrupted by Reasoning: Reasoning Language Models Become Free-Riders in Public Goods Gamescited by 20
  11. 2025When Ethics and Payoffs Diverge: LLM Agents in Morally Charged Social Dilemmascited by 14
  12. 2025Generative Value Conflicts Reveal LLM Prioritiescited by 13
14 more
  1. 2025Societal and technological progress as sewing an ever-growing, ever-changing, patchy, and polychrome quiltcited by 9
  2. 2024Democratic AI is Possible. The Democracy Levels Framework Shows How It Might Workcited by 8
  3. 2025Societal Alignment Frameworks Can Improve LLM Alignmentcited by 8
  4. 2026More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaborationcited by 7
  5. 2025Reputation as a Solution to Cooperation Collapse in LLM-based MASscited by 6
  6. 2025Investigating Prosocial Behavior Theory in LLM Agents under Policy-Induced Inequitiescited by 6
  7. 2025Full-Stack Alignment: Co-Aligning AI and Institutions with Thick Models of Valuecited by 6
  8. 2026Strategies of cooperation and defection in five large language modelscited by 4
  9. 2026A Theory of Appropriateness That Accounts for Norms of Rationalitycited by 4
  10. 2025The Subtle Art of Defection: Understanding Uncooperative Behaviors in LLM based Multi-Agent Systemscited by 3
  11. 2026AgentCity: Constitutional Governance for Autonomous Agent Economies via Separation of Powercited by 3
  12. 2026Evaluating Cooperation in LLM Social Groups through Elected Leadershipcited by 3
  13. 2025Maximizing Social Welfare with Side Paymentscited by 2
  14. 2025Democracy-in-Silico: Institutional Design as Alignment in AI-Governed Politiescited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026LLM Constitutional Multi-Agent Governance
  2. 2026Governance by Design: A Parsonian Institutional Architecture for Internet-Wide Agent Societies
  3. 2026I Can't Believe It's Corrupt: Evaluating Corruption in Multi-Agent Governance Systems
  4. 2026When Agents Evolve, Institutions Follow
  5. 2025Static Sandboxes Are Inadequate: Modeling Societal Complexity Requires Open-Ended Co-Evolution in LLM-Based Multi-Agent Simulations
  6. 2026Population Physics, Population Problems: Safety and Emergence in LLM Societies
  7. 2026Emergence World: A Platform for Evaluating Long-Horizon Multi-Agent Autonomy
  8. 2025Understanding LLM Agent Behaviours via Game Theory: Strategy Recognition, Biases and Multi-Agent Dynamics