# Review of Generative AI Methods in Cybersecurity

arXiv: https://arxiv.org/abs/2403.08701 · HTML: https://mlsanity.com/paper/2403.08701/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 42 of its 95 references that are in the map.
- First posted on arXiv 13 Mar 2024.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 30 of 42)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [Multi-step Jailbreaking Privacy Attacks on ChatGPT](https://mlsanity.com/paper/2304.05197/) (2023)
- [Mathematical Capabilities of ChatGPT](https://mlsanity.com/paper/2301.13867/) (2023)
- [From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy](https://mlsanity.com/paper/2307.00691/) (2023)
- [An Analysis of the Automatic Bug Fixing Performance of ChatGPT](https://mlsanity.com/paper/2301.08653/) (2023)
- [PentestGPT: An LLM-empowered Automatic Penetration Testing Tool](https://mlsanity.com/paper/2308.06782/) (2023)
- [Getting pwn'd by AI: Penetration Testing with Large Language Models](https://mlsanity.com/paper/2308.00121/) (2023)
- [LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Plugins](https://mlsanity.com/paper/2309.10254/) (2023)
- [Revolutionizing Cyber Threat Detection with Large Language Models: A privacy-preserving BERT-based Lightweight Model for IoT/IIoT Devices](https://mlsanity.com/paper/2306.14263/) (2023)
- [The FormAI Dataset: Generative AI in Software Security Through the Lens of Formal Verification](https://mlsanity.com/paper/2307.02192/) (2023)
- [The Ethics of Interaction: Mitigating Security Threats in LLMs](https://mlsanity.com/paper/2401.12273/) (2024)
- [From Text to MITRE Techniques: Exploring the Malicious Use of Large Language Models for Generating Cyber Attack Payloads](https://mlsanity.com/paper/2305.15336/) (2023)
- [MathAttack: Attacking Large Language Models Towards Math Solving Ability](https://mlsanity.com/paper/2309.01686/) (2023)
- [A Machine Learning-oriented Survey on Tiny Machine Learning](https://mlsanity.com/paper/2309.11932/) (2023)
- [LLMs as Hackers: Autonomous Linux Privilege Escalation Attacks](https://mlsanity.com/paper/2310.11409/) (2023)
- [A Survey on Machine Learning Techniques for Source Code Analysis](https://mlsanity.com/paper/2110.09610/) (2021)
- [Decoding the Threat Landscape : ChatGPT, FraudGPT, and WormGPT in Social Engineering Attacks](https://mlsanity.com/paper/2310.05595/) (2023)
- [From GPT-4 to Gemini and Beyond: Assessing the Landscape of MLLMs on Generalizability, Trustworthiness and Causality through Four Modalities](https://mlsanity.com/paper/2401.15071/) (2024)
- [ChatGPT for Digital Forensic Investigation: The Good, The Bad, and The Unknown](https://mlsanity.com/paper/2307.10195/) (2023)
- [Generating Phishing Attacks using ChatGPT](https://mlsanity.com/paper/2305.05133/) (2023)
- [AGIR: Automating Cyber Threat Intelligence Reporting with Natural Language Generation](https://mlsanity.com/paper/2310.02655/) (2023)
- [Gemini in Reasoning: Unveiling Commonsense in Multimodal Large Language Models](https://mlsanity.com/paper/2312.17661/) (2023)
- [Ethical ChatGPT: Concerns, Challenges, and Commandments](https://mlsanity.com/paper/2305.10646/) (2023)
- [Sweeping Heterogeneity with Smart MoPs: Mixture of Prompts for LLM Task Adaptation](https://mlsanity.com/paper/2310.02842/) (2023)

## Built on by (most cited first, 9 of 9)

- [When LLMs Meet Cybersecurity: A Systematic Literature Review](https://mlsanity.com/paper/2405.03644/) (2024)
- [Large Language Models for Cyber Security: A Systematic Literature Review](https://mlsanity.com/paper/2405.04760/) (2024)
- [Benchmarking Practices in LLM-driven Offensive Security: Testbeds, Metrics, and Experiment Design](https://mlsanity.com/paper/2504.10112/) (2025)
- [AthenaBench: A Dynamic Benchmark for Evaluating LLMs in Cyber Threat Intelligence](https://mlsanity.com/paper/2511.01144/) (2025)
- [Securing Generative AI Agentic Workflows: Risks, Mitigation, and a Proposed Firewall Architecture](https://mlsanity.com/paper/2506.17266/) (2025)
- [Contextualized AI for Cyber Defense: An Automated Survey using LLMs](https://mlsanity.com/paper/2409.13524/) (2024)
- [Cyber Deception: State of the art, Trends and Open challenges](https://mlsanity.com/paper/2409.07194/) (2024)
- [Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense Techniques](https://mlsanity.com/paper/2507.13629/) (2025)
- [Agentic and Generative AI for Open-Source Intelligence and Cyber Investigations: Taxonomy, Evaluation, Challenges, and Future Directions](https://mlsanity.com/paper/2607.03233/) (2026)

## Abstract (arXiv)

Over the last decade, Artificial Intelligence (AI) has become increasingly popular, especially with the use of chatbots such as ChatGPT, Gemini, and DALL-E. With this rise, large language models (LLMs) and Generative AI (GenAI) have also become more prevalent in everyday use. These advancements strengthen cybersecurity's defensive posture and open up new attack avenues for adversaries as well. This paper provides a comprehensive overview of the current state-of-the-art deployments of GenAI, covering assaults, jailbreaking, and applications of prompt injection and reverse psychology. This paper also provides the various applications of GenAI in cybercrimes, such as automated hacking, phishing emails, social engineering, reverse cryptography, creating attack payloads, and creating malware. GenAI can significantly improve the automation of defensive cyber security processes through strategies such as dataset construction, safe code development, threat intelligence, defensive measures, reporting, and cyberattack detection. In this study, we suggest that future research should focus on developing robust ethical norms and innovative defense mechanisms to address the current issues that GenAI creates and to also further encourage an impartial approach to its future application in cybersecurity. Moreover, we underscore the importance of interdisciplinary approaches further to bridge the gap between scientific developments and ethical considerations.