# Digital Deception: Generative Artificial Intelligence in Social Engineering and Phishing

arXiv: https://arxiv.org/abs/2310.13715 · HTML: https://mlsanity.com/paper/2310.13715/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2023.

## Built on by (most cited first, 27 of 27)

- [Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents](https://mlsanity.com/paper/2505.02077/) (2025)
- [Mapping the Ethics of Generative AI: A Comprehensive Scoping Review](https://mlsanity.com/paper/2402.08323/) (2024)
- [Defending Against Social Engineering Attacks in the Age of LLMs](https://mlsanity.com/paper/2406.12263/) (2024)
- [Personalized Attacks of Social Engineering in Multi-turn Conversations: LLM Agents for Simulation and Detection](https://mlsanity.com/paper/2503.15552/) (2025)
- [Lateral Phishing With Large Language Models: A Large Organization Comparative Study](https://mlsanity.com/paper/2401.09727/) (2024)
- [Evaluating LLM-based Personal Information Extraction and Countermeasures](https://mlsanity.com/paper/2408.07291/) (2024)
- [AI Awareness](https://mlsanity.com/paper/2504.20084/) (2025)
- [PEAR: Planner-Executor Agent Robustness Benchmark](https://mlsanity.com/paper/2510.07505/) (2025)
- [The AI Model Risk Catalog: What Developers and Researchers Miss About Real-World AI Harms](https://mlsanity.com/paper/2508.16672/) (2025)
- [Gen-AI for User Safety: A Survey](https://mlsanity.com/paper/2411.06606/) (2024)
- [DIRF: A Framework for Digital Identity Protection and Clone Governance in Agentic AI Systems](https://mlsanity.com/paper/2508.01997/) (2025)
- [Assessing AI-Generated vs. Human-Authored Spear Phishing SMS Attacks: An Empirical Study](https://mlsanity.com/paper/2406.13049/) (2024)
- [Psychological Profiling in Cybersecurity: A Look at LLMs and Psycholinguistic Features](https://mlsanity.com/paper/2406.18783/) (2024)
- [Strategic Integration of Artificial Intelligence in the C-Suite: The Role of the Chief AI Officer](https://mlsanity.com/paper/2407.10247/) (2024)
- [Application of AI-based Models for Online Fraud Detection and Analysis](https://mlsanity.com/paper/2409.19022/) (2024)
- [Cyber Shadows: Neutralizing Security Threats with AI and Targeted Policy Measures](https://mlsanity.com/paper/2501.09025/) (2025)
- [Position: It's Time to Act on the Risk of Efficient Personalized Text Generation](https://mlsanity.com/paper/2502.06560/) (2025)
- [Unintentional Consequences: Generative AI Use for Cybercrime](https://mlsanity.com/paper/2505.23733/) (2025)
- [Selfie-Capture Dynamics as an Auxiliary Signal Against Deepfakes and Injection Attacks for Mobile Identity Verification](https://mlsanity.com/paper/2605.00218/) (2026)
- [SAGE: An LLM-driven Self Reflective Agentic Framework for Fraud Detection](https://mlsanity.com/paper/2606.08146/) (2026)
- [Quantum Algorithms: A New Frontier in Financial Crime Prevention](https://mlsanity.com/paper/2403.18322/) (2024)
- [AI Security Map: Holistic Organization of AI Security Technologies and Impacts on Stakeholders](https://mlsanity.com/paper/2508.08583/) (2025)
- ["I'm happy even though it's not real": GenAI Photo Editing as a Remembering Experience](https://mlsanity.com/paper/2602.03104/) (2026)
- [LLM Wardens: Mitigating Adversarial Persuasion with Third-Party Conversational Oversight](https://mlsanity.com/paper/2605.08321/) (2026)
- [Agentic Relationship Harm: Benchmarking and Gating Relational Manipulation in AI Agents](https://mlsanity.com/paper/2606.03271/) (2026)
- [AI in Cyberpsychology: A systematic literature review of Cybersecurity enhancement by using AI for analyzing psychology of Victims, Attackers, and Defenders](https://mlsanity.com/paper/2607.13123/) (2026)
- [Beyond Detection: Evaluating Defensive LLMs Against AI-Generated Social Engineering in Live Turn-by-Turn Interaction](https://mlsanity.com/paper/2608.10239/) (2026)

## Abstract (arXiv)

The advancement of Artificial Intelligence (AI) and Machine Learning (ML) has profound implications for both the utility and security of our digital interactions. This paper investigates the transformative role of Generative AI in Social Engineering (SE) attacks. We conduct a systematic review of social engineering and AI capabilities and use a theory of social engineering to identify three pillars where Generative AI amplifies the impact of SE attacks: Realistic Content Creation, Advanced Targeting and Personalization, and Automated Attack Infrastructure. We integrate these elements into a conceptual model designed to investigate the complex nature of AI-driven SE attacks - the Generative AI Social Engineering Framework. We further explore human implications and potential countermeasures to mitigate these risks. Our study aims to foster a deeper understanding of the risks, human implications, and countermeasures associated with this emerging paradigm, thereby contributing to a more secure and trustworthy human-computer interaction.