I Can't Believe It's Corrupt: Evaluating Corruption in Multi-Agent Governance Systems
On the map: LLM agents and tool use → Social simulation and multi-agent debate
Abstract (arXiv)
Large language models are increasingly proposed as autonomous agents for high-stakes public workflows, yet we lack systematic evidence about whether they would follow institutional rules when granted authority. We present evidence that integrity in institutional AI should be treated as a pre-deployment requirement rather than a post-deployment assumption. We evaluate multi-agent governance simulations in which agents occupy formal governmental roles under different authority structures, and we score rule-breaking and abuse outcomes with an independent rubric-based judge across 28,112 transcript segments. While we advance this position, the core contribution is empirical: among models operating below saturation, governance structure is a stronger driver of corruption-related outcomes than model identity, with large differences across regimes and model--governance pairings. Lightweight safeguards can reduce risk in some settings but do not consistently prevent severe failures. These results imply that institutional design is a precondition for safe delegation: before real authority is assigned to LLM agents, systems should undergo stress testing under governance-like constraints with enforceable rules, auditable logs, and human oversight on high-impact actions.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 8 of its 33 references that are in the map. First posted on arXiv 19 Mar 2026.
Builds on
8 of 33 references.
Closest ancestors · 2
Same direction, newest first
- 2026Talk, Judge, Cooperate: Gossip-Driven Indirect Reciprocity in Self-Interested LLM Agents8 Feb
- 2023Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia6 Dec
Other references · 6
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2016Concrete Problems in AI Safetycited by 1,582
- 2017Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 206
- 2020Open Problems in Cooperative AIcited by 167
- 2025Moloch's Bargain: Emergent Misalignment When LLMs Compete for Audiencescited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026When Agents Evolve, Institutions Follow
- 2026The Containment Gap: How Deployed Agentic AI Frameworks Fail Public-Facing Safety Requirements
- 2025Governance-as-a-Service: A Multi-Agent Framework for AI System Compliance and Policy Enforcement
- 2025Democracy-in-Silico: Institutional Design as Alignment in AI-Governed Polities
- 2026From Certain Doom to Survival: Agent-Driven Self-Governance in LLM Agent Societies
- 2025AGENTSAFE: A Unified Framework for Ethical Assurance and Governance in Agentic AI
- 2026From Safety Risk to Design Principle: Peer-Preservation in Multi-Agent LLM Systems and Its Implications for Orchestrated Democratic Discourse Analysis
- 2025Oversight Structures for Agentic AI in Public-Sector Organizations