The End of Trust: How Agentic AI Breaks Security Assumptions
On the map: LLM agents and tool use → Social simulation and multi-agent debate
Abstract (arXiv)
For decades, the security of digital interaction has rested on an unacknowledged economic constraint. Attackers faced a tradeoff between the fidelity of a deception and the scale at which it could be deployed. Convincing impersonation required sustained human effort and was confined to a narrow set of high-value targets, while mass-market attacks sacrificed plausibility for reach. Detection systems, verification mechanisms, and user awareness training have all been implicitly calibrated to the artifacts of cheap deception that this tradeoff produced. Agentic AI collapses the tradeoff, allowing high-fidelity, individually tailored deception to be produced at mass-market scale. We argue that this shift exhausts a security paradigm rather than merely intensifying the threat landscape. We introduce the Infinite Impostor, an attack model in which an autonomous agent interposes itself between two parties who already trust each other, hijacking an existing relationship rather than building a new one from scratch. Detection-oriented defenses share an assumption that generative progress is eliminating, that synthetic outputs are distinguishable from authentic ones. We propose a suspect-by-default paradigm that shifts security from authenticating actors to evaluating actions, and examine the governance tensions that arise when platforms become the regulatory substrate of digital interaction.
Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 21 of its 82 references that are in the map. First posted on arXiv 14 May 2026.
Builds on
21 of 82 references.
References · 21
- 2023A Watermark for Large Language Modelscited by 717
- 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 717
- 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defensecited by 423
- 2023Can AI-Generated Text be Reliably Detected?cited by 383
- 2020The Creation and Detection of Deepfakes: A Surveycited by 246
- 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 186
- 2023Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigationscited by 157
- 2022Human heuristics for AI-generated language are flawedcited by 105
- 2019Deep Learning for Deepfakes Creation and Detection: A Surveycited by 102
- 2023Spear Phishing With Large Language Modelscited by 51
- 2022Seeing is Living? Rethinking the Security of Facial Liveness Verification in the Deepfake Eracited by 26
- 2023ChatGPT: More than a Weapon of Mass Deception, Ethical challenges and responses from the Human-Centered Artificial Intelligence (HCAI) perspectivecited by 22
9 more
- 2024Evaluating Large Language Models' Capability to Launch Fully Automated Spear Phishing Campaigns: Validated on Human Subjectscited by 17
- 2024Personhood credentials: Artificial intelligence and the value of privacy-preserving tools to distinguish who is real onlinecited by 16
- 2024Defending Against Social Engineering Attacks in the Age of LLMscited by 15
- 2025Watermarking Without Standards Is Not AI Governancecited by 9
- 2019Better Safe Than Sorry: An Adversarial Approach to Improve Social Bot Detectioncited by 8
- 2024Topic-Based Watermarks for Large Language Modelscited by 8
- 2024Content, Nudges and Incentives: A Study on the Effectiveness and Perception of Embedded Phishing Trainingcited by 5
- 2021Phishing in Organizations: Findings from a Large-Scale and Long-Term Studycited by 4
- 2024The Cat and Mouse Game: The Ongoing Arms Race Between Diffusion Models and Detection Methodscited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026I must delete the evidence: AI Agents Explicitly Cover up Fraud and Violent Crime
- 2026Deception and Communication in Autonomous Multi-Agent Systems: An Experimental Study with Among Us
- 2025The Traitors: Deception and Trust in Multi-Agent Language Model Simulations
- 2026AgentKernel: The Trust-Native Agentic Operating System
- 2026Agentic Societies Need a Social Harness
- 2026Lies We Can See: Joint Verbal and Non-Verbal Deception by VLM Agents in Embodied Social Interactions
- 2026DecepEval: A Benchmark for Evaluating Deception in LLM Agents
- 2026Disclosure By Design: Identity Transparency as a Behavioural Property of Conversational AI Models