mlsanityData to 14 Oct 2026

Review of Generative AI Methods in Cybersecurity

arXiv 2403.08701 ↗First posted 13 Mar 2024cs.CRcited by 9

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

Over the last decade, Artificial Intelligence (AI) has become increasingly popular, especially with the use of chatbots such as ChatGPT, Gemini, and DALL-E. With this rise, large language models (LLMs) and Generative AI (GenAI) have also become more prevalent in everyday use. These advancements strengthen cybersecurity's defensive posture and open up new attack avenues for adversaries as well. This paper provides a comprehensive overview of the current state-of-the-art deployments of GenAI, covering assaults, jailbreaking, and applications of prompt injection and reverse psychology. This paper also provides the various applications of GenAI in cybercrimes, such as automated hacking, phishing emails, social engineering, reverse cryptography, creating attack payloads, and creating malware. GenAI can significantly improve the automation of defensive cyber security processes through strategies such as dataset construction, safe code development, threat intelligence, defensive measures, reporting, and cyberattack detection. In this study, we suggest that future research should focus on developing robust ethical norms and innovative defense mechanisms to address the current issues that GenAI creates and to also further encourage an impartial approach to its future application in cybersecurity. Moreover, we underscore the importance of interdisciplinary approaches further to bridge the gap between scientific developments and ethical considerations.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 42 of its 95 references that are in the map. First posted on arXiv 13 Mar 2024.

Builds on

42 of 95 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy3 Jul
  2. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPT11 Apr

Other references · 40

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  6. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  7. 2023A Comprehensive Overview of Large Language Modelscited by 648
  8. 2023Mathematical Capabilities of ChatGPTcited by 268
  9. 2023An Analysis of the Automatic Bug Fixing Performance of ChatGPTcited by 110
  10. 2023PentestGPT: An LLM-empowered Automatic Penetration Testing Toolcited by 97
  11. 2023Getting pwn'd by AI: Penetration Testing with Large Language Modelscited by 83
  12. 2023LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Pluginscited by 51
28 more
  1. 2023Revolutionizing Cyber Threat Detection with Large Language Models: A privacy-preserving BERT-based Lightweight Model for IoT/IIoT Devicescited by 39
  2. 2023The FormAI Dataset: Generative AI in Software Security Through the Lens of Formal Verificationcited by 34
  3. 2024The Ethics of Interaction: Mitigating Security Threats in LLMscited by 33
  4. 2023From Text to MITRE Techniques: Exploring the Malicious Use of Large Language Models for Generating Cyber Attack Payloadscited by 32
  5. 2023MathAttack: Attacking Large Language Models Towards Math Solving Abilitycited by 32
  6. 2023A Machine Learning-oriented Survey on Tiny Machine Learningcited by 27
  7. 2023LLMs as Hackers: Autonomous Linux Privilege Escalation Attackscited by 27
  8. 2021A Survey on Machine Learning Techniques for Source Code Analysiscited by 24
  9. 2023Decoding the Threat Landscape : ChatGPT, FraudGPT, and WormGPT in Social Engineering Attackscited by 23
  10. 2024From GPT-4 to Gemini and Beyond: Assessing the Landscape of MLLMs on Generalizability, Trustworthiness and Causality through Four Modalitiescited by 19
  11. 2023ChatGPT for Digital Forensic Investigation: The Good, The Bad, and The Unknowncited by 17
  12. 2023Generating Phishing Attacks using ChatGPTcited by 16
  13. 2023AGIR: Automating Cyber Threat Intelligence Reporting with Natural Language Generationcited by 16
  14. 2023Gemini in Reasoning: Unveiling Commonsense in Multimodal Large Language Modelscited by 16
  15. 2023Ethical ChatGPT: Concerns, Challenges, and Commandmentscited by 14
  16. 2023Sweeping Heterogeneity with Smart MoPs: Mixture of Prompts for LLM Task Adaptationcited by 12
  17. 2023On the Uses of Large Language Models to Interpret Ambiguous Cyberattack Descriptionscited by 11
  18. 2023Exploring the Dark Side of AI: Advanced Phishing Attack Design and Deployment Using ChatGPTcited by 9
  19. 2023Deployment Corrections: An incident response framework for frontier AI modelscited by 9
  20. 2021Cyber Threat Intelligence Model: An Evaluation of Taxonomies, Sharing Standards, and Ontologies within Cyber Threat Intelligencecited by 8
  21. 2023Maximizing Penetration Testing Success with Effective Reconnaissance Techniques using ChatGPTcited by 6
  22. 2022Multi-Level Fine-Tuning, Data Augmentation, and Few-Shot Learning for Specialized Cyber Threat Intelligencecited by 4
  23. 2023Digital Twin-Empowered Smart Attack Detection System for 6G Edge of Things Networkscited by 4
  24. 2023Digital Twin-Enabled Intelligent DDoS Detection Mechanism for Autonomous Core Networkscited by 3
  25. 2023MetaAID 2.5: A Secure Framework for Developing Metaverse Applications via Large Language Modelscited by 3
  26. 2023Building Resilient SMEs: Harnessing Large Language Models for Cyber Security in Australiacited by 2
  27. 2023TwinPot: Digital Twin-assisted Honeypot for Cyber-Secure Smart Seaportscited by 2
  28. 2023ChatGPT vs. Lightweight Security: First Work Implementing the NIST Cryptographic Standard ASCONcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 4; 2026: 1.

9 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025How Generative AI Empowers Attackers and Defenders Across the Trust & Safety Landscape
  2. 2025Jailbreaking Generative AI: Multivector Phishing Threats and Transformer based Defenses
  3. 2026Large Language Models (LLMs) and Generative AI in Cybersecurity and Privacy: A Survey of Dual-Use Risks, AI-Generated Malware, Explainability, and Defensive Strategies
  4. 2025GenAI Security: Outsmarting the Bots with a Proactive Testing Framework
  5. 2023Identifying and Mitigating the Security Risks of Generative AI
  6. 2024Generative AI in Cybersecurity: A Comprehensive Review of LLM Applications and Vulnerabilities
  7. 2025Jailbreaking Generative AI: Empowering Novices to Conduct Phishing Attacks
  8. 2024Poison Attacks and Adversarial Prompts Against an Informed University Virtual Assistant