# Next-Generation Phishing: How LLM Agents Empower Cyber Attackers

arXiv: https://arxiv.org/abs/2411.13874 · HTML: https://mlsanity.com/paper/2411.13874/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 23 references that are in the map.
- First posted on arXiv 21 Nov 2024.

## Builds on (most cited first, 14 of 14)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications](https://mlsanity.com/paper/2402.07927/) (2024)
- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks](https://mlsanity.com/paper/2302.05733/) (2023)
- [LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models](https://mlsanity.com/paper/2309.12307/) (2023)
- [The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning](https://mlsanity.com/paper/2205.03401/) (2022)
- [A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systems](https://mlsanity.com/paper/2402.18649/) (2024)
- [Spear Phishing With Large Language Models](https://mlsanity.com/paper/2305.06972/) (2023)
- [ChatSpamDetector: Leveraging Large Language Models for Effective Phishing Email Detection](https://mlsanity.com/paper/2402.18093/) (2024)
- [Spam-T5: Benchmarking Large Language Models for Few-Shot Email Spam Detection](https://mlsanity.com/paper/2304.01238/) (2023)

## Built on by (most cited first, 14 of 14)

- [The Phish, The Spam, and The Valid: Generating Feature-Rich Emails for Benchmarking LLMs](https://mlsanity.com/paper/2511.21448/) (2025)
- [LLM-Stackelberg Games: Conjectural Reasoning Equilibria and Their Applications to Spearphishing](https://mlsanity.com/paper/2507.09407/) (2025)
- [On the Feasibility of Using MultiModal LLMs to Execute AR Social Engineering Attacks](https://mlsanity.com/paper/2504.13209/) (2025)
- [Can LLMs Help Allocate Public Health Resources? A Case Study on Childhood Lead Testing](https://mlsanity.com/paper/2511.18239/) (2025)
- [Evaluating Large Language Models for Phishing Detection, Self-Consistency, Faithfulness, and Explainability](https://mlsanity.com/paper/2506.13746/) (2025)
- [Phishing Detection in the Gen-AI Era: Quantized LLMs vs Classical Models](https://mlsanity.com/paper/2507.07406/) (2025)
- [SCOUT: A Defense Against Data Poisoning Attacks in Fine-Tuned Language Models](https://mlsanity.com/paper/2512.10998/) (2025)
- [UNSEEN: A Cross-Stack LLM Unlearning Defense against AR-LLM Social Engineering Attacks](https://mlsanity.com/paper/2604.23141/) (2026)
- [PhySE: A Psychological Framework for Real-Time AR-LLM Social Engineering Attacks](https://mlsanity.com/paper/2604.23148/) (2026)
- [CAREBench: A Child-Safety Risk Benchmark for Language Models](https://mlsanity.com/paper/2606.29685/) (2026)
- [Assessing Spear-Phishing Website Generation in Large Language Model Coding Agents](https://mlsanity.com/paper/2602.13363/) (2026)
- [Algospeak, Hiding in the Open: The Trade-off Between Legible Meaning and Detection Avoidance](https://mlsanity.com/paper/2605.06619/) (2026)
- [Can LLM Agents Simulate Dynamic Networks? A Case Study on Email Networks with Phishing Synthesis](https://mlsanity.com/paper/2605.12507/) (2026)
- [AURA: Adaptive Uncertainty-Routed Analysis for Email Threat Detection](https://mlsanity.com/paper/2609.19873/) (2026)

## Abstract (arXiv)

The escalating threat of phishing emails has become increasingly sophisticated with the rise of Large Language Models (LLMs). As attackers exploit LLMs to craft more convincing and evasive phishing emails, it is crucial to assess the resilience of current phishing defenses. In this study we conduct a comprehensive evaluation of traditional phishing detectors, such as Gmail Spam Filter, Apache SpamAssassin, and Proofpoint, as well as machine learning models like SVM, Logistic Regression, and Naive Bayes, in identifying both traditional and LLM-rephrased phishing emails. We also explore the emerging role of LLMs as phishing detection tools, a method already adopted by companies like NTT Security Holdings and JPMorgan Chase. Our results reveal notable declines in detection accuracy for rephrased emails across all detectors, highlighting critical weaknesses in current phishing defenses. As the threat landscape evolves, our findings underscore the need for stronger security controls and regulatory oversight on LLM-generated content to prevent its misuse in creating advanced phishing attacks. This study contributes to the development of more effective Cyber Threat Intelligence (CTI) by leveraging LLMs to generate diverse phishing variants that can be used for data augmentation, harnessing the power of LLMs to enhance phishing detection, and paving the way for more robust and adaptable threat detection systems.