# The Traitors: Deception and Trust in Multi-Agent Language Model Simulations

arXiv: https://arxiv.org/abs/2505.12923 · HTML: https://mlsanity.com/paper/2505.12923/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 21 of its 69 references that are in the map.
- First posted on arXiv 19 May 2025.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 21 of 21)

- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [Scalable agent alignment via reward modeling: a research direction](https://mlsanity.com/paper/1811.07871/) (2018)
- [AI safety via debate](https://mlsanity.com/paper/1805.00899/) (2018)
- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- [Teaching language models to support answers with verified quotes](https://mlsanity.com/paper/2203.11147/) (2022)
- [Risks from Learned Optimization in Advanced Machine Learning Systems](https://mlsanity.com/paper/1906.01820/) (2019)
- [Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf](https://mlsanity.com/paper/2309.04658/) (2023)
- [Language Models as Agent Models](https://mlsanity.com/paper/2212.01681/) (2022)
- [Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents](https://mlsanity.com/paper/2404.16698/) (2024)
- [Truthful AI: Developing and governing AI that does not lie](https://mlsanity.com/paper/2110.06674/) (2021)
- [Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplation](https://mlsanity.com/paper/2310.01320/) (2023)
- [Reinforcement Learning with a Corrupted Reward Channel](https://mlsanity.com/paper/1705.08417/) (2017)
- [Emergent Communication through Negotiation](https://mlsanity.com/paper/1804.03980/) (2018)
- [X-Risk Analysis for AI Research](https://mlsanity.com/paper/2206.05862/) (2022)
- [OpenToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Models](https://mlsanity.com/paper/2402.06044/) (2024)
- [Hidden Incentives for Auto-Induced Distributional Shift](https://mlsanity.com/paper/2009.09153/) (2020)
- [Decentralised Learning in Systems with Many, Many Strategic Agents](https://mlsanity.com/paper/1803.05028/) (2018)
- [Among Them: A game-based framework for assessing persuasion capabilities of LLMs](https://mlsanity.com/paper/2502.20426/) (2025)
- [Persuasion Games using Large Language Models](https://mlsanity.com/paper/2408.15879/) (2024)

## Built on by (most cited first, 30 of 34)

- [Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond](https://mlsanity.com/paper/2604.22748/) (2026)
- [WOLF: Werewolf-based Observations for LLM Deception and Falsehoods](https://mlsanity.com/paper/2512.09187/) (2025)
- [Build Your Personalized Research Group: A Multiagent Framework for Continual and Interactive Science Automation](https://mlsanity.com/paper/2510.15624/) (2025)
- [Detecting Multi-Agent Collusion Through Multi-Agent Interpretability](https://mlsanity.com/paper/2604.01151/) (2026)
- [Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation](https://mlsanity.com/paper/2606.10749/) (2026)
- [TrinityGuard: A Unified Framework for Safeguarding Multi-Agent Systems](https://mlsanity.com/paper/2603.15408/) (2026)
- [Emergent Risks in Generative Multi-Agent Systems](https://mlsanity.com/paper/2603.27771/) (2026)
- [AgentReputation: A Decentralized Agentic AI Reputation Framework](https://mlsanity.com/paper/2605.00073/) (2026)
- [Extending the OWASP Multi-Agentic System Threat Modeling Guide: Insights from Multi-Agent Security Research](https://mlsanity.com/paper/2508.09815/) (2025)
- [Many-to-One Adversarial Consensus: Exposing Multi-Agent Collusion Risks in AI-Based Healthcare](https://mlsanity.com/paper/2512.03097/) (2025)
- [CRAFT: Grounded Multi-Agent Coordination Under Partial Information](https://mlsanity.com/paper/2603.25268/) (2026)
- [QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agents](https://mlsanity.com/paper/2605.27068/) (2026)
- [Building and Measuring Trust between Large Language Models](https://mlsanity.com/paper/2508.15858/) (2025)
- [Shall We Play a Game? Language Models for Open-ended Wargames](https://mlsanity.com/paper/2509.17192/) (2025)
- [Multi-Agent Framework for Threat Mitigation and Resilience in AI-Based Systems](https://mlsanity.com/paper/2512.23132/) (2025)
- [The AI in the Mirror: LLM Self-Recognition in an Iterated Public Goods Game](https://mlsanity.com/paper/2508.18467/) (2025)
- [Unified Threat Detection and Mitigation Framework (UTDMF): Combating Prompt Injection, Deception, and Bias in Enterprise-Scale Transformers](https://mlsanity.com/paper/2510.04528/) (2025)
- [Towards Ethical Multi-Agent Systems of Large Language Models: A Mechanistic Interpretability Perspective](https://mlsanity.com/paper/2512.04691/) (2025)
- [Emergent Strategic Reasoning Risks in AI: A Taxonomy-Driven Evaluation Framework](https://mlsanity.com/paper/2604.22119/) (2026)
- [GAMBIT: A Three-Mode Benchmark for Adversarial Robustness in Multi-Agent LLM Collectives](https://mlsanity.com/paper/2605.09027/) (2026)
- [Evaluating Large Language Models in a Complex Hidden Role Game](https://mlsanity.com/paper/2605.22826/) (2026)
- [RogueAI: A Reverse Turing Test for Detecting Licensed AI Deception in Dialogue](https://mlsanity.com/paper/2606.13310/) (2026)
- [What to Ask Next? Probing the Imaginative Reasoning of LLMs with TurtleSoup Puzzles](https://mlsanity.com/paper/2508.10358/) (2025)
- [Intentional Deception as Controllable Capability in LLM Agents](https://mlsanity.com/paper/2603.07848/) (2026)
- [Stable Reasoning, Unstable Responses: Mitigating LLM Deception via Stability Asymmetry](https://mlsanity.com/paper/2603.26846/) (2026)
- [SMAC-Talk: A Natural Language Extension of the StarCraft Multi-Agent Challenge for Large Language Models](https://mlsanity.com/paper/2606.04202/) (2026)
- [Triadic Werewolf: A Jester Role for Multi-Hop Theory of Mind in LLMs](https://mlsanity.com/paper/2606.27909/) (2026)
- [Is Lying an Emergent Behaviour in LLMs? Evidence from Gaslighting AI agents in a Sustainability Game](https://mlsanity.com/paper/2606.28456/) (2026)
- [When Agents Lie: Premeditation, Persistence, and Exploitation in Repeated Games](https://mlsanity.com/paper/2607.05132/) (2026)
- [Can Agents Deceive? Evaluating Reasoning and Deception in ParliamentBench using a Social Deduction Game](https://mlsanity.com/paper/2607.28146/) (2026)

## Abstract (arXiv)

As AI systems increasingly assume roles where trust and alignment with human values are essential, understanding when and why they engage in deception has become a critical research priority. We introduce The Traitors, a multi-agent simulation framework inspired by social deduction games, designed to probe deception, trust formation, and strategic communication among large language model (LLM) agents under asymmetric information. A minority of agents the traitors seek to mislead the majority, while the faithful must infer hidden identities through dialogue and reasoning. Our contributions are: (1) we ground the environment in formal frameworks from game theory, behavioral economics, and social cognition; (2) we develop a suite of evaluation metrics capturing deception success, trust dynamics, and collective inference quality; (3) we implement a fully autonomous simulation platform where LLMs reason over persistent memory and evolving social dynamics, with support for heterogeneous agent populations, specialized traits, and adaptive behaviors. Our initial experiments across DeepSeek-V3, GPT-4o-mini, and GPT-4o (10 runs per model) reveal a notable asymmetry: advanced models like GPT-4o demonstrate superior deceptive capabilities yet exhibit disproportionate vulnerability to others' falsehoods. This suggests deception skills may scale faster than detection abilities. Overall, The Traitors provides a focused, configurable testbed for investigating LLM behavior in socially nuanced interactions. We position this work as a contribution toward more rigorous research on deception mechanisms, alignment challenges, and the broader social reliability of AI systems.