# Large Language Models in Cybersecurity: State-of-the-Art

arXiv: https://arxiv.org/abs/2402.00891 · HTML: https://mlsanity.com/paper/2402.00891/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 102 references that are in the map.
- First posted on arXiv 30 Jan 2024.

## Builds on (most cited first, 30 of 54)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [Towards Reasoning in Large Language Models: A Survey](https://mlsanity.com/paper/2212.10403/) (2022)
- [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://mlsanity.com/paper/2304.13712/) (2023)
- [Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction](https://mlsanity.com/paper/2005.12833/) (2020)
- [A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT](https://mlsanity.com/paper/2302.09419/) (2023)
- [HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledge](https://mlsanity.com/paper/2304.06975/) (2023)
- [Large Language Models for Code: Security Hardening and Adversarial Testing](https://mlsanity.com/paper/2302.05319/) (2023)
- [In-Context Impersonation Reveals Large Language Models' Strengths and Biases](https://mlsanity.com/paper/2305.14930/) (2023)
- [Examining Zero-Shot Vulnerability Repair with Large Language Models](https://mlsanity.com/paper/2112.02125/) (2021)
- [PentestGPT: An LLM-empowered Automatic Penetration Testing Tool](https://mlsanity.com/paper/2308.06782/) (2023)
- [XuanYuan 2.0: A Large Chinese Financial Chat Model with Hundreds of Billions Parameters](https://mlsanity.com/paper/2305.12002/) (2023)
- [Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants](https://mlsanity.com/paper/2208.09727/) (2022)
- [Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Tasks](https://mlsanity.com/paper/2305.14201/) (2023)
- [Getting pwn'd by AI: Penetration Testing with Large Language Models](https://mlsanity.com/paper/2308.00121/) (2023)
- [Large Language Models for Supply Chain Optimization](https://mlsanity.com/paper/2307.03875/) (2023)
- [Automating Customer Service using LangChain: Building custom open-source GPT Chatbot for organizations](https://mlsanity.com/paper/2310.05421/) (2023)
- [SecureBERT: A Domain-Specific Language Model for Cybersecurity](https://mlsanity.com/paper/2204.02685/) (2022)
- [AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model](https://mlsanity.com/paper/2208.01448/) (2022)
- [Privacy in Large Language Models: Attacks, Defenses and Future Directions](https://mlsanity.com/paper/2310.10383/) (2023)
- [Training Data Extraction From Pre-trained Language Models: A Survey](https://mlsanity.com/paper/2305.16157/) (2023)
- [MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response](https://mlsanity.com/paper/2309.08730/) (2023)
- [A New Era in Software Security: Towards Self-Healing Software via Large Language Models and Formal Verification](https://mlsanity.com/paper/2305.14752/) (2023)
- [When ChatGPT Meets Smart Contract Vulnerability Detection: How Far Are We?](https://mlsanity.com/paper/2309.05520/) (2023)

## Built on by (most cited first, 30 of 34)

- [When LLMs Meet Cybersecurity: A Systematic Literature Review](https://mlsanity.com/paper/2405.03644/) (2024)
- [Large Language Models for Cyber Security: A Systematic Literature Review](https://mlsanity.com/paper/2405.04760/) (2024)
- [HackSynth: LLM Agent and Evaluation Framework for Autonomous Penetration Testing](https://mlsanity.com/paper/2412.01778/) (2024)
- [Llama-3.1-FoundationAI-SecurityLLM-8B-Instruct Technical Report](https://mlsanity.com/paper/2508.01059/) (2025)
- [ProFLingo: A Fingerprinting-based Intellectual Property Protection Scheme for Large Language Models](https://mlsanity.com/paper/2405.02466/) (2024)
- [CyberRAG: An Agentic RAG cyber attack classification and reporting tool](https://mlsanity.com/paper/2507.02424/) (2025)
- [Transfer Learning in Pre-Trained Large Language Models for Malware Detection Based on System Calls](https://mlsanity.com/paper/2405.09318/) (2024)
- [Can Language Models Falsify? Evaluating Algorithmic Reasoning with Counterexample Creation](https://mlsanity.com/paper/2502.19414/) (2025)
- [Distributed Threat Intelligence at the Edge Devices: A Large Language Model-Driven Approach](https://mlsanity.com/paper/2405.08755/) (2024)
- [BreachSeek: A Multi-Agent Automated Penetration Tester](https://mlsanity.com/paper/2409.03789/) (2024)
- [A Unified Framework for Human AI Collaboration in Security Operations Centers with Trusted Autonomy](https://mlsanity.com/paper/2505.23397/) (2025)
- [CmdCaliper: A Semantic-Aware Command-Line Embedding Model and Dataset for Security Research](https://mlsanity.com/paper/2411.01176/) (2024)
- [Large Multimodal Agents for Accurate Phishing Detection with Enhanced Token Optimization and Cost Reduction](https://mlsanity.com/paper/2412.02301/) (2024)
- [SpecPipe: Accelerating Pipeline Parallelism-based LLM Inference with Speculative Decoding](https://mlsanity.com/paper/2504.04104/) (2025)
- [Toward Cybersecurity-Expert Small Language Models](https://mlsanity.com/paper/2510.14113/) (2025)
- [HULLMI: Human vs LLM identification with explainability](https://mlsanity.com/paper/2409.04808/) (2024)
- [OMNISEC: LLM-Driven Provenance-based Intrusion Detection via Retrieval-Augmented Behavior Prompting](https://mlsanity.com/paper/2503.03108/) (2025)
- [AutoBnB-RAG: Enhancing Multi-Agent Incident Response with Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.13118/) (2025)
- [SoK: Are Watermarks in LLMs Ready for Deployment?](https://mlsanity.com/paper/2506.05594/) (2025)
- [Securing Educational LLMs: A Generalised Taxonomy of Attacks on LLMs and DREAD Risk Assessment](https://mlsanity.com/paper/2508.08629/) (2025)
- [Adapting Large Language Models to Emerging Cybersecurity using Retrieval Augmented Generation](https://mlsanity.com/paper/2510.27080/) (2025)
- [The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelines](https://mlsanity.com/paper/2512.06659/) (2025)
- [Psychological Profiling in Cybersecurity: A Look at LLMs and Psycholinguistic Features](https://mlsanity.com/paper/2406.18783/) (2024)
- [Trustworthy GNNs with LLMs: A Systematic Review and Taxonomy](https://mlsanity.com/paper/2502.08353/) (2025)
- [Vulnerability Mitigation System (VMS): LLM Agent and Evaluation Framework for Autonomous Penetration Testing](https://mlsanity.com/paper/2507.21113/) (2025)
- [LanG -- A Governance-Aware Agentic AI Platform for Unified Security Operations](https://mlsanity.com/paper/2604.05440/) (2026)
- [Post-Hoc Trajectory-Risk Certification for Modular LLM-Based Security Agents](https://mlsanity.com/paper/2608.05199/) (2026)
- [Beyond Trusting Trust: Multi-Model Validation for Robust Code Generation](https://mlsanity.com/paper/2502.16279/) (2025)
- [How Good LLM-Generated Password Policies Are?](https://mlsanity.com/paper/2506.08320/) (2025)
- [CyGATE: Game-Theoretic Cyber Attack-Defense Engine for Patch Strategy Optimization](https://mlsanity.com/paper/2508.00478/) (2025)

## Abstract (arXiv)

The rise of Large Language Models (LLMs) has revolutionized our comprehension of intelligence bringing us closer to Artificial Intelligence. Since their introduction, researchers have actively explored the applications of LLMs across diverse fields, significantly elevating capabilities. Cybersecurity, traditionally resistant to data-driven solutions and slow to embrace machine learning, stands out as a domain. This study examines the existing literature, providing a thorough characterization of both defensive and adversarial applications of LLMs within the realm of cybersecurity. Our review not only surveys and categorizes the current landscape but also identifies critical research gaps. By evaluating both offensive and defensive applications, we aim to provide a holistic understanding of the potential risks and opportunities associated with LLM-driven cybersecurity.