Review of Generative AI Methods in Cybersecurity
On the map: LLM safety and security → Prompt injection and agent security
Abstract (arXiv)
Over the last decade, Artificial Intelligence (AI) has become increasingly popular, especially with the use of chatbots such as ChatGPT, Gemini, and DALL-E. With this rise, large language models (LLMs) and Generative AI (GenAI) have also become more prevalent in everyday use. These advancements strengthen cybersecurity's defensive posture and open up new attack avenues for adversaries as well. This paper provides a comprehensive overview of the current state-of-the-art deployments of GenAI, covering assaults, jailbreaking, and applications of prompt injection and reverse psychology. This paper also provides the various applications of GenAI in cybercrimes, such as automated hacking, phishing emails, social engineering, reverse cryptography, creating attack payloads, and creating malware. GenAI can significantly improve the automation of defensive cyber security processes through strategies such as dataset construction, safe code development, threat intelligence, defensive measures, reporting, and cyberattack detection. In this study, we suggest that future research should focus on developing robust ethical norms and innovative defense mechanisms to address the current issues that GenAI creates and to also further encourage an impartial approach to its future application in cybersecurity. Moreover, we underscore the importance of interdisciplinary approaches further to bridge the gap between scientific developments and ethical considerations.
Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 42 of its 95 references that are in the map. First posted on arXiv 13 Mar 2024.
Builds on
42 of 95 references.
Closest ancestors · 2
Same direction, newest first
- 2023From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy3 Jul
- 2023Multi-step Jailbreaking Privacy Attacks on ChatGPT11 Apr
Other references · 40
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023GPT-4 Technical Reportcited by 18,396
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
- 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
- 2023A Comprehensive Overview of Large Language Modelscited by 648
- 2023Mathematical Capabilities of ChatGPTcited by 268
- 2023An Analysis of the Automatic Bug Fixing Performance of ChatGPTcited by 110
- 2023PentestGPT: An LLM-empowered Automatic Penetration Testing Toolcited by 97
- 2023Getting pwn'd by AI: Penetration Testing with Large Language Modelscited by 83
- 2023LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Pluginscited by 51
28 more
- 2023Revolutionizing Cyber Threat Detection with Large Language Models: A privacy-preserving BERT-based Lightweight Model for IoT/IIoT Devicescited by 39
- 2023The FormAI Dataset: Generative AI in Software Security Through the Lens of Formal Verificationcited by 34
- 2024The Ethics of Interaction: Mitigating Security Threats in LLMscited by 33
- 2023From Text to MITRE Techniques: Exploring the Malicious Use of Large Language Models for Generating Cyber Attack Payloadscited by 32
- 2023MathAttack: Attacking Large Language Models Towards Math Solving Abilitycited by 32
- 2023A Machine Learning-oriented Survey on Tiny Machine Learningcited by 27
- 2023LLMs as Hackers: Autonomous Linux Privilege Escalation Attackscited by 27
- 2021A Survey on Machine Learning Techniques for Source Code Analysiscited by 24
- 2023Decoding the Threat Landscape : ChatGPT, FraudGPT, and WormGPT in Social Engineering Attackscited by 23
- 2024From GPT-4 to Gemini and Beyond: Assessing the Landscape of MLLMs on Generalizability, Trustworthiness and Causality through Four Modalitiescited by 19
- 2023ChatGPT for Digital Forensic Investigation: The Good, The Bad, and The Unknowncited by 17
- 2023Generating Phishing Attacks using ChatGPTcited by 16
- 2023AGIR: Automating Cyber Threat Intelligence Reporting with Natural Language Generationcited by 16
- 2023Gemini in Reasoning: Unveiling Commonsense in Multimodal Large Language Modelscited by 16
- 2023Ethical ChatGPT: Concerns, Challenges, and Commandmentscited by 14
- 2023Sweeping Heterogeneity with Smart MoPs: Mixture of Prompts for LLM Task Adaptationcited by 12
- 2023On the Uses of Large Language Models to Interpret Ambiguous Cyberattack Descriptionscited by 11
- 2023Exploring the Dark Side of AI: Advanced Phishing Attack Design and Deployment Using ChatGPTcited by 9
- 2023Deployment Corrections: An incident response framework for frontier AI modelscited by 9
- 2021Cyber Threat Intelligence Model: An Evaluation of Taxonomies, Sharing Standards, and Ontologies within Cyber Threat Intelligencecited by 8
- 2023Maximizing Penetration Testing Success with Effective Reconnaissance Techniques using ChatGPTcited by 6
- 2022Multi-Level Fine-Tuning, Data Augmentation, and Few-Shot Learning for Specialized Cyber Threat Intelligencecited by 4
- 2023Digital Twin-Empowered Smart Attack Detection System for 6G Edge of Things Networkscited by 4
- 2023Digital Twin-Enabled Intelligent DDoS Detection Mechanism for Autonomous Core Networkscited by 3
- 2023MetaAID 2.5: A Secure Framework for Developing Metaverse Applications via Large Language Modelscited by 3
- 2023Building Resilient SMEs: Harnessing Large Language Models for Cyber Security in Australiacited by 2
- 2023TwinPot: Digital Twin-assisted Honeypot for Cyber-Secure Smart Seaportscited by 2
- 2023ChatGPT vs. Lightweight Security: First Work Implementing the NIST Cryptographic Standard ASCONcited by 1
Built on by
Citing papers per year: 2024: 4; 2025: 4; 2026: 1.
9 papers.
- 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 71
- 2024Large Language Models for Cyber Security: A Systematic Literature Reviewcited by 68
- 2025Benchmarking Practices in LLM-driven Offensive Security: Testbeds, Metrics, and Experiment Designcited by 14
- 2025AthenaBench: A Dynamic Benchmark for Evaluating LLMs in Cyber Threat Intelligencecited by 4
- 2025Securing Generative AI Agentic Workflows: Risks, Mitigation, and a Proposed Firewall Architecturecited by 2
- 2024Contextualized AI for Cyber Defense: An Automated Survey using LLMscited by 1
- 2024Cyber Deception: State of the art, Trends and Open challengescited by 0
- 2025Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense Techniquescited by 0
- 2026Agentic and Generative AI for Open-Source Intelligence and Cyber Investigations: Taxonomy, Evaluation, Challenges, and Future Directionscited by 0
- 2026Agentic and Generative AI for Open-Source Intelligence and Cyber Investigations: Taxonomy, Evaluation, Challenges, and Future Directions3 Jul 2026
- 2025AthenaBench: A Dynamic Benchmark for Evaluating LLMs in Cyber Threat Intelligence3 Nov 2025
- 2025Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense Techniques18 Jul 2025
- 2025Securing Generative AI Agentic Workflows: Risks, Mitigation, and a Proposed Firewall Architecture10 Jun 2025
- 2025Benchmarking Practices in LLM-driven Offensive Security: Testbeds, Metrics, and Experiment Design14 Apr 2025
- 2024Contextualized AI for Cyber Defense: An Automated Survey using LLMs20 Sep 2024
- 2024Cyber Deception: State of the art, Trends and Open challenges11 Sep 2024
- 2024Large Language Models for Cyber Security: A Systematic Literature Review8 May 2024
- 2024When LLMs Meet Cybersecurity: A Systematic Literature Review6 May 2024
Similar, not cited
Close in wording, no citation either way
- 2025How Generative AI Empowers Attackers and Defenders Across the Trust & Safety Landscape
- 2025Jailbreaking Generative AI: Multivector Phishing Threats and Transformer based Defenses
- 2026Large Language Models (LLMs) and Generative AI in Cybersecurity and Privacy: A Survey of Dual-Use Risks, AI-Generated Malware, Explainability, and Defensive Strategies
- 2025GenAI Security: Outsmarting the Bots with a Proactive Testing Framework
- 2023Identifying and Mitigating the Security Risks of Generative AI
- 2024Generative AI in Cybersecurity: A Comprehensive Review of LLM Applications and Vulnerabilities
- 2025Jailbreaking Generative AI: Empowering Novices to Conduct Phishing Attacks
- 2024Poison Attacks and Adversarial Prompts Against an Informed University Virtual Assistant