When Agents Lie: Premeditation, Persistence, and Exploitation in Repeated Games
On the map: LLM agents and tool use → Social simulation and multi-agent debate
Abstract (arXiv)
As large language models are deployed as autonomous agents that communicate intentions before acting, a critical safety question is whether agents that publicly commit to actions will honor those commitments. We place LLM agents in repeated n-player games with a three-stage protocol that separates private intent, public announcement, and final action, allowing us to identify whether each deviation from a stated announcement was already planned during private deliberation. Evaluating three frontier models across six games in homogeneous and heterogeneous groups over 10 rounds, we report two findings. First, when agents deviate from their announcements, the deviation is predominantly already stated in their private plan (exceeding 90% in the highest-deception conditions), yet this is not a fixed model property: the same model ranges from perfect honesty to near-total deviation across games. Second, different models interpret announcements incompatibly, some as binding commitments and others as cheap talk, producing payoff gaps that emerge in Round~0 and persist across all 10 rounds. Systems that combine models from different providers therefore cannot assume shared announcement semantics and require empirical testing of model interactions before deployment.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 25 of its 39 references that are in the map. First posted on arXiv 6 Jul 2026.
Builds on
25 of 39 references.
Closest ancestors · 6
Same direction, newest first
- 2026CoopEval: Benchmarking Cooperation-Sustaining Mechanisms and LLM Agents in Social Dilemmas16 Apr
- 2026Deception and Communication in Autonomous Multi-Agent Systems: An Experimental Study with Among Us27 Mar
- 2026LieCraft: A Multi-Agent Framework for Evaluating Deceptive Capabilities in Language Models6 Mar
- 2025The Traitors: Deception and Trust in Multi-Agent Language Model Simulations19 May
- 2025Simulating Cooperative Prosocial Behavior with Multi-Agent LLMs: Evidence and Mechanisms for AI Agents to Inform Policy Decisions18 Feb
- 2024How Well Can LLMs Negotiate? NegotiationArena Platform and Analysis8 Feb
Other references · 19
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
- 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
- 2024Alignment faking in large language modelscited by 360
- 2024Frontier Models are Capable of In-context Schemingcited by 287
- 2023Playing repeated games with Large Language Modelscited by 217
- 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 188
- 2023Deception Abilities Emerged in Large Language Modelscited by 101
- 2023Honesty Is the Best Policy: Defining and Mitigating AI Deceptioncited by 40
- 2025When Thinking LLMs Lie: Unveiling the Strategic Deception in Representations of Reasoning Modelscited by 18
- 2025When Ethics and Payoffs Diverge: LLM Agents in Morally Charged Social Dilemmascited by 15
- 2025WOLF: Werewolf-based Observations for LLM Deception and Falsehoodscited by 13
7 more
- 2025When Do Large Language Models Exhibit Unsolicited Deception?cited by 10
- 2025LLM Bidders Preserve the Mechanism-Level Orderings of Human Bidderscited by 10
- 2025Evaluating & Reducing Deceptive Dialogue From Language Models with Multi-turn RLcited by 10
- 2025Deceive, Detect, and Disclose: Large Language Models Play Mini-Mafiacited by 9
- 2026GT-HarmBench: Benchmarking AI Safety Risks Through the Lens of Game Theorycited by 5
- 2026Cheap Talk, Empty Promise: Frontier LLMs easily break public promises for self-interestcited by 3
- 2024Evaluating Creativity and Deception in Large Language Models: A Simulation Framework for Multi-Agent Balderdashcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Be Careful Who You Trust: Coordination Dynamics under Corrupted Communication in LLM Multi-Agent Games
- 2026Cheap Talk Stabilizes Strategic Interaction in LLM Agents
- 2026Even More Deception: Objective Misalignment in Mixed-Motive LLM Multi-Agent Systems
- 2026Knowledge-Verified Emergent Deception in LLM Agents Under Conflicting Incentives
- 2026Hidden in Plain Text: Measuring LLM Deception Quality Against Human Baselines Using Social Deduction Games
- 2026Lying to Win: Assessing LLM Deception through Human-AI Games and Parallel-World Probing
- 2026DecepEval: A Benchmark for Evaluating Deception in LLM Agents
- 2026The Politician, the Liar, and the Obedient Worker: Emerging Behavior of LLM Agents in Hierarchical Games