A Lifecycle and Application-Stack Survey of Large Language Model Vulnerabilities: Attacks, Risks, Defenses, and Open Problems
Not on the map. Why?
Abstract (arXiv)
Large language models are no longer only text generators. They are increasingly embedded in retrieval pipelines, enterprise assistants, coding environments, robotic systems, security-operation workflows, and autonomous agents that can read private data, call tools, write files, execute code, and act across organizational boundaries. This shift changes the security problem: risks do not arise from the model weights alone, but from the full lifecycle and application stack through which data, prompts, model outputs, tools, memories, and user authority interact. This paper systematizes the literature on vulnerabilities in large language model systems through a lifecycle and application-stack lens. We organize attacks across eight stages: data collection, pretraining, post-training alignment, model packaging and supply chain, retrieval and memory, prompting and inference, tool/agent execution, and deployment/maintenance. For each stage, we analyze attacker capabilities, affected security objectives, representative attacks, practical risks, evaluation practices, and defenses. We further map LLM-specific vulnerabilities to confidentiality, integrity, availability, safety, privacy, fairness, accountability, and agency-control objectives. Unlike taxonomies that list isolated attack names, the proposed systematization emphasizes where trust boundaries fail, how untrusted data becomes executable instruction, how delegated authority amplifies model errors, and why point defenses rarely compose. We close with a research agenda for secure LLM systems, including compositional security, provenance-aware retrieval, tool-call containment, long-horizon agent evaluation, privacy-preserving adaptation, realistic red teaming, and deployment-grade incident response.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 45 of its 64 references that are in the map. First posted on arXiv 30 Jun 2026.
Builds on
45 of 64 references.
References · 45
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
33 more
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2016Deep Learning with Differential Privacycited by 2,682
- 2020Learning to summarize from human feedbackcited by 2,562
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
- 2020Extracting Training Data from Large Language Modelscited by 2,144
- 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
- 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
- 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
- 2021Ethical and social risks of harm from Language Modelscited by 957
- 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
- 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
- 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
- 2023A Comprehensive Overview of Large Language Modelscited by 639
- 2016Stealing Machine Learning Models via Prediction APIscited by 602
- 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
- 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
- 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
- 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
- 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
- 2023Poisoning Language Models During Instruction Tuningcited by 254
- 2024AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agentscited by 129
- 2023PentestGPT: An LLM-empowered Automatic Penetration Testing Toolcited by 94
- 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflowscited by 57
- 2026ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injectioncited by 25
- 2026SoK: The Attack Surface of Agentic AI - Tools and Autonomycited by 14
- 2025A Survey of Attacks on Large Language Modelscited by 13
- 2026A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Frameworkcited by 9
Built on by
Not cited yet.