The Traitors: Deception and Trust in Multi-Agent Language Model Simulations
On the map: LLM agents and tool use → Social simulation and multi-agent debate
Abstract (arXiv)
As AI systems increasingly assume roles where trust and alignment with human values are essential, understanding when and why they engage in deception has become a critical research priority. We introduce The Traitors, a multi-agent simulation framework inspired by social deduction games, designed to probe deception, trust formation, and strategic communication among large language model (LLM) agents under asymmetric information. A minority of agents the traitors seek to mislead the majority, while the faithful must infer hidden identities through dialogue and reasoning. Our contributions are: (1) we ground the environment in formal frameworks from game theory, behavioral economics, and social cognition; (2) we develop a suite of evaluation metrics capturing deception success, trust dynamics, and collective inference quality; (3) we implement a fully autonomous simulation platform where LLMs reason over persistent memory and evolving social dynamics, with support for heterogeneous agent populations, specialized traits, and adaptive behaviors. Our initial experiments across DeepSeek-V3, GPT-4o-mini, and GPT-4o (10 runs per model) reveal a notable asymmetry: advanced models like GPT-4o demonstrate superior deceptive capabilities yet exhibit disproportionate vulnerability to others' falsehoods. This suggests deception skills may scale faster than detection abilities. Overall, The Traitors provides a focused, configurable testbed for investigating LLM behavior in socially nuanced interactions. We position this work as a contribution toward more rigorous research on deception mechanisms, alignment challenges, and the broader social reliability of AI systems.
Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 21 of its 69 references that are in the map. First posted on arXiv 19 May 2025.
Builds on
21 of 69 references.
Closest ancestors · 1
Same direction, newest first
Other references · 20
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
- 2016Concrete Problems in AI Safetycited by 1,642
- 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
- 2018Scalable agent alignment via reward modeling: a research directioncited by 447
- 2018AI safety via debatecited by 400
- 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
- 2022Teaching language models to support answers with verified quotescited by 292
- 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 258
- 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 241
- 2022Language Models as Agent Modelscited by 132
- 2021Truthful AI: Developing and governing AI that does not liecited by 113
- 2023Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplationcited by 104
8 more
- 2017Reinforcement Learning with a Corrupted Reward Channelcited by 70
- 2018Emergent Communication through Negotiationcited by 62
- 2022X-Risk Analysis for AI Researchcited by 60
- 2024OpenToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Modelscited by 54
- 2020Hidden Incentives for Auto-Induced Distributional Shiftcited by 43
- 2018Decentralised Learning in Systems with Many, Many Strategic Agentscited by 20
- 2025Among Them: A game-based framework for assessing persuasion capabilities of LLMscited by 8
- 2024Persuasion Games using Large Language Modelscited by 4
Built on by
Citing papers per year: 2025: 11; 2026: 23.
34 papers.
Sort:
By area:
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 14
- 2025WOLF: Werewolf-based Observations for LLM Deception and Falsehoodscited by 13
- 2025Build Your Personalized Research Group: A Multiagent Framework for Continual and Interactive Science Automationcited by 9
- 2026Detecting Multi-Agent Collusion Through Multi-Agent Interpretabilitycited by 7
- 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 7
- 2026TrinityGuard: A Unified Framework for Safeguarding Multi-Agent Systemscited by 5
- 2026Emergent Risks in Generative Multi-Agent Systemscited by 5
- 2026AgentReputation: A Decentralized Agentic AI Reputation Frameworkcited by 4
- 2025Extending the OWASP Multi-Agentic System Threat Modeling Guide: Insights from Multi-Agent Security Researchcited by 3
- 2025Many-to-One Adversarial Consensus: Exposing Multi-Agent Collusion Risks in AI-Based Healthcarecited by 3
- 2026CRAFT: Grounded Multi-Agent Coordination Under Partial Informationcited by 3
- 2026QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agentscited by 3
22 more
- 2025Building and Measuring Trust between Large Language Modelscited by 2
- 2025Shall We Play a Game? Language Models for Open-ended Wargamescited by 2
- 2025Multi-Agent Framework for Threat Mitigation and Resilience in AI-Based Systemscited by 2
- 2025The AI in the Mirror: LLM Self-Recognition in an Iterated Public Goods Gamecited by 1
- 2025Unified Threat Detection and Mitigation Framework (UTDMF): Combating Prompt Injection, Deception, and Bias in Enterprise-Scale Transformerscited by 1
- 2025Towards Ethical Multi-Agent Systems of Large Language Models: A Mechanistic Interpretability Perspectivecited by 1
- 2026Emergent Strategic Reasoning Risks in AI: A Taxonomy-Driven Evaluation Frameworkcited by 1
- 2026GAMBIT: A Three-Mode Benchmark for Adversarial Robustness in Multi-Agent LLM Collectivescited by 1
- 2026Evaluating Large Language Models in a Complex Hidden Role Gamecited by 1
- 2026RogueAI: A Reverse Turing Test for Detecting Licensed AI Deception in Dialoguecited by 1
- 2025What to Ask Next? Probing the Imaginative Reasoning of LLMs with TurtleSoup Puzzlescited by 0
- 2026Intentional Deception as Controllable Capability in LLM Agentscited by 0
- 2026Stable Reasoning, Unstable Responses: Mitigating LLM Deception via Stability Asymmetrycited by 0
- 2026SMAC-Talk: A Natural Language Extension of the StarCraft Multi-Agent Challenge for Large Language Modelscited by 0
- 2026Triadic Werewolf: A Jester Role for Multi-Hop Theory of Mind in LLMscited by 0
- 2026Is Lying an Emergent Behaviour in LLMs? Evidence from Gaslighting AI agents in a Sustainability Gamecited by 0
- 2026When Agents Lie: Premeditation, Persistence, and Exploitation in Repeated Gamescited by 0
- 2026Can Agents Deceive? Evaluating Reasoning and Deception in ParliamentBench using a Social Deduction Gamecited by 0
- 2026AI Alignment through a Game-theoretic Lens: A Surveycited by 0
- 2026Lies We Can See: Joint Verbal and Non-Verbal Deception by VLM Agents in Embodied Social Interactionscited by 0
- 2026Be Careful Who You Trust: Coordination Dynamics under Corrupted Communication in LLM Multi-Agent Gamescited by 0
- 2026DecepEval: A Benchmark for Evaluating Deception in LLM Agentscited by 0
- 2026DecepEval: A Benchmark for Evaluating Deception in LLM Agents6 Oct 2026
- 2026Be Careful Who You Trust: Coordination Dynamics under Corrupted Communication in LLM Multi-Agent Games19 Sep 2026
- 2026Lies We Can See: Joint Verbal and Non-Verbal Deception by VLM Agents in Embodied Social Interactions31 Aug 2026
- 2026AI Alignment through a Game-theoretic Lens: A Survey28 Aug 2026
- 2026Can Agents Deceive? Evaluating Reasoning and Deception in ParliamentBench using a Social Deduction Game30 Jul 2026
- 2026When Agents Lie: Premeditation, Persistence, and Exploitation in Repeated Games6 Jul 2026
- 2026Is Lying an Emergent Behaviour in LLMs? Evidence from Gaslighting AI agents in a Sustainability Game26 Jun 2026
- 2026Triadic Werewolf: A Jester Role for Multi-Hop Theory of Mind in LLMs26 Jun 2026
- 2026RogueAI: A Reverse Turing Test for Detecting Licensed AI Deception in Dialogue11 Jun 2026
- 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation9 Jun 2026
- 2026SMAC-Talk: A Natural Language Extension of the StarCraft Multi-Agent Challenge for Large Language Models2 Jun 2026
- 2026QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agents26 May 2026
22 more
- 2026GAMBIT: A Three-Mode Benchmark for Adversarial Robustness in Multi-Agent LLM Collectives9 May 2026
- 2026AgentReputation: A Decentralized Agentic AI Reputation Framework30 Apr 2026
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond24 Apr 2026
- 2026Emergent Strategic Reasoning Risks in AI: A Taxonomy-Driven Evaluation Framework23 Apr 2026
- 2026Evaluating Large Language Models in a Complex Hidden Role Game9 Apr 2026
- 2026Detecting Multi-Agent Collusion Through Multi-Agent Interpretability1 Apr 2026
- 2026Emergent Risks in Generative Multi-Agent Systems29 Mar 2026
- 2026Stable Reasoning, Unstable Responses: Mitigating LLM Deception via Stability Asymmetry27 Mar 2026
- 2026CRAFT: Grounded Multi-Agent Coordination Under Partial Information26 Mar 2026
- 2026TrinityGuard: A Unified Framework for Safeguarding Multi-Agent Systems16 Mar 2026
- 2026Intentional Deception as Controllable Capability in LLM Agents8 Mar 2026
- 2025Multi-Agent Framework for Threat Mitigation and Resilience in AI-Based Systems29 Dec 2025
- 2025WOLF: Werewolf-based Observations for LLM Deception and Falsehoods9 Dec 2025
- 2025Towards Ethical Multi-Agent Systems of Large Language Models: A Mechanistic Interpretability Perspective4 Dec 2025
- 2025Many-to-One Adversarial Consensus: Exposing Multi-Agent Collusion Risks in AI-Based Healthcare1 Dec 2025
- 2025Build Your Personalized Research Group: A Multiagent Framework for Continual and Interactive Science Automation17 Oct 2025
- 2025Unified Threat Detection and Mitigation Framework (UTDMF): Combating Prompt Injection, Deception, and Bias in Enterprise-Scale Transformers6 Oct 2025
- 2025Shall We Play a Game? Language Models for Open-ended Wargames21 Sep 2025
- 2025The AI in the Mirror: LLM Self-Recognition in an Iterated Public Goods Game25 Aug 2025
- 2025Building and Measuring Trust between Large Language Models20 Aug 2025
- 2025What to Ask Next? Probing the Imaginative Reasoning of LLMs with TurtleSoup Puzzles14 Aug 2025
- 2025Extending the OWASP Multi-Agentic System Threat Modeling Guide: Insights from Multi-Agent Security Research13 Aug 2025
- 2026Detecting Multi-Agent Collusion Through Multi-Agent Interpretabilitycited by 7
- 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 7
- 2026TrinityGuard: A Unified Framework for Safeguarding Multi-Agent Systemscited by 5
- 2026AgentReputation: A Decentralized Agentic AI Reputation Frameworkcited by 4
- 2025Extending the OWASP Multi-Agentic System Threat Modeling Guide: Insights from Multi-Agent Security Researchcited by 3
- 2025Many-to-One Adversarial Consensus: Exposing Multi-Agent Collusion Risks in AI-Based Healthcarecited by 3
- 2026CRAFT: Grounded Multi-Agent Coordination Under Partial Informationcited by 3
- 2026QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agentscited by 3
- 2025Multi-Agent Framework for Threat Mitigation and Resilience in AI-Based Systemscited by 2
- 2025Towards Ethical Multi-Agent Systems of Large Language Models: A Mechanistic Interpretability Perspectivecited by 1
- 2026GAMBIT: A Three-Mode Benchmark for Adversarial Robustness in Multi-Agent LLM Collectivescited by 1
- 2026Intentional Deception as Controllable Capability in LLM Agentscited by 0
12 most cited of 18 All 18 (JSON)
- 2026DecepEval: A Benchmark for Evaluating Deception in LLM Agents6 Oct 2026
- 2026Be Careful Who You Trust: Coordination Dynamics under Corrupted Communication in LLM Multi-Agent Games19 Sep 2026
- 2026Lies We Can See: Joint Verbal and Non-Verbal Deception by VLM Agents in Embodied Social Interactions31 Aug 2026
- 2026When Agents Lie: Premeditation, Persistence, and Exploitation in Repeated Games6 Jul 2026
- 2026Is Lying an Emergent Behaviour in LLMs? Evidence from Gaslighting AI agents in a Sustainability Game26 Jun 2026
- 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation9 Jun 2026
- 2026SMAC-Talk: A Natural Language Extension of the StarCraft Multi-Agent Challenge for Large Language Models2 Jun 2026
- 2026QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agents26 May 2026
- 2026GAMBIT: A Three-Mode Benchmark for Adversarial Robustness in Multi-Agent LLM Collectives9 May 2026
- 2026AgentReputation: A Decentralized Agentic AI Reputation Framework30 Apr 2026
- 2026Detecting Multi-Agent Collusion Through Multi-Agent Interpretability1 Apr 2026
- 2026CRAFT: Grounded Multi-Agent Coordination Under Partial Information26 Mar 2026
12 newest of 18 All 18 (JSON)
- 2026Emergent Strategic Reasoning Risks in AI: A Taxonomy-Driven Evaluation Frameworkcited by 1
- 2025What to Ask Next? Probing the Imaginative Reasoning of LLMs with TurtleSoup Puzzlescited by 0
- 2026Stable Reasoning, Unstable Responses: Mitigating LLM Deception via Stability Asymmetrycited by 0
- 2026Can Agents Deceive? Evaluating Reasoning and Deception in ParliamentBench using a Social Deduction Gamecited by 0
- 2026Can Agents Deceive? Evaluating Reasoning and Deception in ParliamentBench using a Social Deduction Game30 Jul 2026
- 2026Emergent Strategic Reasoning Risks in AI: A Taxonomy-Driven Evaluation Framework23 Apr 2026
- 2026Stable Reasoning, Unstable Responses: Mitigating LLM Deception via Stability Asymmetry27 Mar 2026
- 2025What to Ask Next? Probing the Imaginative Reasoning of LLMs with TurtleSoup Puzzles14 Aug 2025
- 2026AI Alignment through a Game-theoretic Lens: A Surveycited by 0
- 2026AI Alignment through a Game-theoretic Lens: A Survey28 Aug 2026
- 2025WOLF: Werewolf-based Observations for LLM Deception and Falsehoodscited by 13
- 2025Build Your Personalized Research Group: A Multiagent Framework for Continual and Interactive Science Automationcited by 9
- 2026Emergent Risks in Generative Multi-Agent Systemscited by 5
- 2025Building and Measuring Trust between Large Language Modelscited by 2
- 2025Shall We Play a Game? Language Models for Open-ended Wargamescited by 2
- 2025The AI in the Mirror: LLM Self-Recognition in an Iterated Public Goods Gamecited by 1
- 2026Evaluating Large Language Models in a Complex Hidden Role Gamecited by 1
- 2026RogueAI: A Reverse Turing Test for Detecting Licensed AI Deception in Dialoguecited by 1
- 2026Triadic Werewolf: A Jester Role for Multi-Hop Theory of Mind in LLMscited by 0
- 2026Triadic Werewolf: A Jester Role for Multi-Hop Theory of Mind in LLMs26 Jun 2026
- 2026RogueAI: A Reverse Turing Test for Detecting Licensed AI Deception in Dialogue11 Jun 2026
- 2026Evaluating Large Language Models in a Complex Hidden Role Game9 Apr 2026
- 2026Emergent Risks in Generative Multi-Agent Systems29 Mar 2026
- 2025WOLF: Werewolf-based Observations for LLM Deception and Falsehoods9 Dec 2025
- 2025Build Your Personalized Research Group: A Multiagent Framework for Continual and Interactive Science Automation17 Oct 2025
- 2025Shall We Play a Game? Language Models for Open-ended Wargames21 Sep 2025
- 2025The AI in the Mirror: LLM Self-Recognition in an Iterated Public Goods Game25 Aug 2025
- 2025Building and Measuring Trust between Large Language Models20 Aug 2025
Similar, not cited
Close in wording, no citation either way
- 2026Deception and Communication in Autonomous Multi-Agent Systems: An Experimental Study with Among Us
- 2026Hidden in Plain Text: Measuring LLM Deception Quality Against Human Baselines Using Social Deduction Games
- 2025Among Us: A Sandbox for Measuring and Detecting Agentic Deception
- 2026LieCraft: A Multi-Agent Framework for Evaluating Deceptive Capabilities in Language Models
- 2024Can Large Language Model Agents Simulate Human Trust Behavior?
- 2026Lying to Win: Assessing LLM Deception through Human-AI Games and Parallel-World Probing
- 2026Trust Between AI Agents: Measuring Formation, Breakage, and Recovery, with Implications for Governing Multi-Agent Systems
- 2026Even More Deception: Objective Misalignment in Mixed-Motive LLM Multi-Agent Systems