mlsanityData to 14 Oct 2026

Evolving Deception: When Agents Evolve, Deception Wins

arXiv 2603.05872 ↗First posted 6 Mar 2026cs.CRcited by 2

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

Self-evolving agents offer a promising path toward scalable autonomy. However, in this work, we show that in competitive environments, self-evolution can instead give rise to a serious and previously underexplored risk: the spontaneous emergence of deception as an evolutionarily stable strategy. We conduct a systematic empirical study on the self-evolution of large language model (LLM) agents in a competitive Bidding Arena, where agents iteratively refine their strategies through interaction-driven reflection. Across different evolutionary paths (eg, Neutral, Honesty-Guided, and Deception-Guided), we find a consistent pattern: under utility-driven competition, unconstrained self-evolution reliably drifts toward deceptive behaviors, even when honest strategies remain viable. This drift is explained by a fundamental asymmetry in generalization. Deception evolves as a transferable meta-strategy that generalizes robustly across diverse and unseen tasks, whereas honesty-based strategies are fragile and often collapse outside their original contexts. Further analysis of agents internal states reveals the emergence of rationalization mechanisms, through which agents justify or deny deceptive actions to reconcile competitive success with normative instructions. Our paper exposes a fundamental tension between agent self-evolution and alignment, highlighting the risks of deploying self-improving agents in adversarial environments.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 20 of its 41 references that are in the map. First posted on arXiv 6 Mar 2026.

Builds on

20 of 41 references.

References · 20

  1. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  2. 2024LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Taskscited by 243
  3. 2024AgentSquare: Automatic LLM Agent Search in Modular Design Spacecited by 155
  4. 2023Large Language Models can Strategically Deceive their Users when Put Under Pressurecited by 146
  5. 2018Transferable Adversarial Attacks for Image and Video Object Detectioncited by 109
  6. 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Promptcited by 109
  7. 2023Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiationcited by 96
  8. 2024VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Modelscited by 87
  9. 2024ToolGen: Unified Tool Retrieval and Calling via Generationcited by 74
  10. 2020Efficient Adversarial Attacks for Visual Object Trackingcited by 66
  11. 2024Semantic Mirror Jailbreak: Genetic Algorithm Based Jailbreak Prompts Against Open-source LLMscited by 53
  12. 2025Manipulating Multimodal Agents via Cross-Modal Prompt Injectioncited by 44
8 more
  1. 2024Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attackscited by 38
  2. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructionscited by 38
  3. 2024Unlearning Backdoor Threats: Enhancing Backdoor Defense in Multimodal Contrastive Learning via Local Token Unlearningcited by 31
  4. 2024Interpreting Object-level Foundation Models via Visual Precision Searchcited by 20
  5. 2025T2VShield: Model-Agnostic Jailbreak Defense for Text-to-Video Modelscited by 18
  6. 2025SafeMobile: Chain-level Jailbreak Detection and Automated Evaluation for Multimodal Mobile Agentscited by 15
  7. 2025RTBAgent: A LLM-based Agent System for Real-Time Biddingcited by 14
  8. 2025CogMorph: Cognitive Morphing Attacks for Text-to-Image Modelscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Deception and Communication in Autonomous Multi-Agent Systems: An Experimental Study with Among Us
  2. 2026DecepEval: A Benchmark for Evaluating Deception in LLM Agents
  3. 2026SAGE: A Quantitative Evaluation of Socialized Evolution in Agent Ecosystems
  4. 2026Trust, Lies, and Long Memories: Emergent Social Dynamics and Reputation in Multi-Round Avalon with LLM Agents
  5. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence
  6. 2026When Agents Lie: Premeditation, Persistence, and Exploitation in Repeated Games
  7. 2026Emergence of Reputation-Based Cooperation in LLM Agents
  8. 2025The Traitors: Deception and Trust in Multi-Agent Language Model Simulations