mlsanityData to 7 Oct 2026

A Lifecycle and Application-Stack Survey of Large Language Model Vulnerabilities: Attacks, Risks, Defenses, and Open Problems

arXiv 2606.31639 ↗First posted 30 Jun 2026cs.CR cs.AI cs.GTcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models are no longer only text generators. They are increasingly embedded in retrieval pipelines, enterprise assistants, coding environments, robotic systems, security-operation workflows, and autonomous agents that can read private data, call tools, write files, execute code, and act across organizational boundaries. This shift changes the security problem: risks do not arise from the model weights alone, but from the full lifecycle and application stack through which data, prompts, model outputs, tools, memories, and user authority interact. This paper systematizes the literature on vulnerabilities in large language model systems through a lifecycle and application-stack lens. We organize attacks across eight stages: data collection, pretraining, post-training alignment, model packaging and supply chain, retrieval and memory, prompting and inference, tool/agent execution, and deployment/maintenance. For each stage, we analyze attacker capabilities, affected security objectives, representative attacks, practical risks, evaluation practices, and defenses. We further map LLM-specific vulnerabilities to confidentiality, integrity, availability, safety, privacy, fairness, accountability, and agency-control objectives. Unlike taxonomies that list isolated attack names, the proposed systematization emphasizes where trust boundaries fail, how untrusted data becomes executable instruction, how delegated authority amplifies model errors, and why point defenses rarely compose. We close with a research agenda for secure LLM systems, including compositional security, provenance-aware retrieval, tool-call containment, long-horizon agent evaluation, privacy-preserving adaptation, realistic red teaming, and deployment-grade incident response.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 45 of its 64 references that are in the map. First posted on arXiv 30 Jun 2026.

Builds on

45 of 64 references.

References · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  7. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  9. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  10. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  11. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  12. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
33 more
  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  3. 2016Deep Learning with Differential Privacycited by 2,682
  4. 2020Learning to summarize from human feedbackcited by 2,562
  5. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  6. 2020Extracting Training Data from Large Language Modelscited by 2,144
  7. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  8. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  9. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  10. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  11. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  12. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  13. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  14. 2021Ethical and social risks of harm from Language Modelscited by 957
  15. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  16. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  17. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  18. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  19. 2023A Comprehensive Overview of Large Language Modelscited by 639
  20. 2016Stealing Machine Learning Models via Prediction APIscited by 602
  21. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  22. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  23. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  24. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
  25. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  26. 2023Poisoning Language Models During Instruction Tuningcited by 254
  27. 2024AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agentscited by 129
  28. 2023PentestGPT: An LLM-empowered Automatic Penetration Testing Toolcited by 94
  29. 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflowscited by 57
  30. 2026ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injectioncited by 25
  31. 2026SoK: The Attack Surface of Agentic AI - Tools and Autonomycited by 14
  32. 2025A Survey of Attacks on Large Language Modelscited by 13
  33. 2026A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Frameworkcited by 9

Built on by

Not cited yet.