mlsanityData to 14 Oct 2026

The Traitors: Deception and Trust in Multi-Agent Language Model Simulations

arXiv 2505.12923 ↗First posted 19 May 2025cs.AI cs.MAcited by 34

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

As AI systems increasingly assume roles where trust and alignment with human values are essential, understanding when and why they engage in deception has become a critical research priority. We introduce The Traitors, a multi-agent simulation framework inspired by social deduction games, designed to probe deception, trust formation, and strategic communication among large language model (LLM) agents under asymmetric information. A minority of agents the traitors seek to mislead the majority, while the faithful must infer hidden identities through dialogue and reasoning. Our contributions are: (1) we ground the environment in formal frameworks from game theory, behavioral economics, and social cognition; (2) we develop a suite of evaluation metrics capturing deception success, trust dynamics, and collective inference quality; (3) we implement a fully autonomous simulation platform where LLMs reason over persistent memory and evolving social dynamics, with support for heterogeneous agent populations, specialized traits, and adaptive behaviors. Our initial experiments across DeepSeek-V3, GPT-4o-mini, and GPT-4o (10 runs per model) reveal a notable asymmetry: advanced models like GPT-4o demonstrate superior deceptive capabilities yet exhibit disproportionate vulnerability to others' falsehoods. This suggests deception skills may scale faster than detection abilities. Overall, The Traitors provides a focused, configurable testbed for investigating LLM behavior in socially nuanced interactions. We position this work as a contribution toward more rigorous research on deception mechanisms, alignment challenges, and the broader social reliability of AI systems.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 21 of its 69 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

21 of 69 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents25 Apr

Other references · 20

  1. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  2. 2016Concrete Problems in AI Safetycited by 1,642
  3. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  4. 2018Scalable agent alignment via reward modeling: a research directioncited by 447
  5. 2018AI safety via debatecited by 400
  6. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
  7. 2022Teaching language models to support answers with verified quotescited by 292
  8. 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 258
  9. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 241
  10. 2022Language Models as Agent Modelscited by 132
  11. 2021Truthful AI: Developing and governing AI that does not liecited by 113
  12. 2023Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplationcited by 104
8 more
  1. 2017Reinforcement Learning with a Corrupted Reward Channelcited by 70
  2. 2018Emergent Communication through Negotiationcited by 62
  3. 2022X-Risk Analysis for AI Researchcited by 60
  4. 2024OpenToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Modelscited by 54
  5. 2020Hidden Incentives for Auto-Induced Distributional Shiftcited by 43
  6. 2018Decentralised Learning in Systems with Many, Many Strategic Agentscited by 20
  7. 2025Among Them: A game-based framework for assessing persuasion capabilities of LLMscited by 8
  8. 2024Persuasion Games using Large Language Modelscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 23.

34 papers.

Sort:

By area:

  1. 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 14
  2. 2025WOLF: Werewolf-based Observations for LLM Deception and Falsehoodscited by 13
  3. 2025Build Your Personalized Research Group: A Multiagent Framework for Continual and Interactive Science Automationcited by 9
  4. 2026Detecting Multi-Agent Collusion Through Multi-Agent Interpretabilitycited by 7
  5. 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 7
  6. 2026TrinityGuard: A Unified Framework for Safeguarding Multi-Agent Systemscited by 5
  7. 2026Emergent Risks in Generative Multi-Agent Systemscited by 5
  8. 2026AgentReputation: A Decentralized Agentic AI Reputation Frameworkcited by 4
  9. 2025Extending the OWASP Multi-Agentic System Threat Modeling Guide: Insights from Multi-Agent Security Researchcited by 3
  10. 2025Many-to-One Adversarial Consensus: Exposing Multi-Agent Collusion Risks in AI-Based Healthcarecited by 3
  11. 2026CRAFT: Grounded Multi-Agent Coordination Under Partial Informationcited by 3
  12. 2026QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agentscited by 3
22 more
  1. 2025Building and Measuring Trust between Large Language Modelscited by 2
  2. 2025Shall We Play a Game? Language Models for Open-ended Wargamescited by 2
  3. 2025Multi-Agent Framework for Threat Mitigation and Resilience in AI-Based Systemscited by 2
  4. 2025The AI in the Mirror: LLM Self-Recognition in an Iterated Public Goods Gamecited by 1
  5. 2025Unified Threat Detection and Mitigation Framework (UTDMF): Combating Prompt Injection, Deception, and Bias in Enterprise-Scale Transformerscited by 1
  6. 2025Towards Ethical Multi-Agent Systems of Large Language Models: A Mechanistic Interpretability Perspectivecited by 1
  7. 2026Emergent Strategic Reasoning Risks in AI: A Taxonomy-Driven Evaluation Frameworkcited by 1
  8. 2026GAMBIT: A Three-Mode Benchmark for Adversarial Robustness in Multi-Agent LLM Collectivescited by 1
  9. 2026Evaluating Large Language Models in a Complex Hidden Role Gamecited by 1
  10. 2026RogueAI: A Reverse Turing Test for Detecting Licensed AI Deception in Dialoguecited by 1
  11. 2025What to Ask Next? Probing the Imaginative Reasoning of LLMs with TurtleSoup Puzzlescited by 0
  12. 2026Intentional Deception as Controllable Capability in LLM Agentscited by 0
  13. 2026Stable Reasoning, Unstable Responses: Mitigating LLM Deception via Stability Asymmetrycited by 0
  14. 2026SMAC-Talk: A Natural Language Extension of the StarCraft Multi-Agent Challenge for Large Language Modelscited by 0
  15. 2026Triadic Werewolf: A Jester Role for Multi-Hop Theory of Mind in LLMscited by 0
  16. 2026Is Lying an Emergent Behaviour in LLMs? Evidence from Gaslighting AI agents in a Sustainability Gamecited by 0
  17. 2026When Agents Lie: Premeditation, Persistence, and Exploitation in Repeated Gamescited by 0
  18. 2026Can Agents Deceive? Evaluating Reasoning and Deception in ParliamentBench using a Social Deduction Gamecited by 0
  19. 2026AI Alignment through a Game-theoretic Lens: A Surveycited by 0
  20. 2026Lies We Can See: Joint Verbal and Non-Verbal Deception by VLM Agents in Embodied Social Interactionscited by 0
  21. 2026Be Careful Who You Trust: Coordination Dynamics under Corrupted Communication in LLM Multi-Agent Gamescited by 0
  22. 2026DecepEval: A Benchmark for Evaluating Deception in LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Deception and Communication in Autonomous Multi-Agent Systems: An Experimental Study with Among Us
  2. 2026Hidden in Plain Text: Measuring LLM Deception Quality Against Human Baselines Using Social Deduction Games
  3. 2025Among Us: A Sandbox for Measuring and Detecting Agentic Deception
  4. 2026LieCraft: A Multi-Agent Framework for Evaluating Deceptive Capabilities in Language Models
  5. 2024Can Large Language Model Agents Simulate Human Trust Behavior?
  6. 2026Lying to Win: Assessing LLM Deception through Human-AI Games and Parallel-World Probing
  7. 2026Trust Between AI Agents: Measuring Formation, Breakage, and Recovery, with Implications for Governing Multi-Agent Systems
  8. 2026Even More Deception: Objective Misalignment in Mixed-Motive LLM Multi-Agent Systems