mlsanityData to 14 Oct 2026

Security Concerns for Large Language Models: A Survey

arXiv 2505.18889 ↗First posted 24 May 2025cs.CR cs.AIcited by 29

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) such as ChatGPT and its competitors have caused a revolution in natural language processing, but their capabilities also introduce new security vulnerabilities. This survey provides a comprehensive overview of these emerging concerns, categorizing threats into several key areas: inference-time attacks via prompt manipulation; training-time attacks; misuse by malicious actors; and the inherent risks in autonomous LLM agents. Recently, a significant focus is increasingly being placed on the latter. We summarize recent academic and industrial studies from 2022 to 2025 that exemplify each threat, analyze existing defense mechanisms and their limitations, and identify open challenges in securing LLM-based applications. We conclude by emphasizing the importance of advancing robust, multi-layered security strategies to ensure LLMs are safe and beneficial.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 63 of its 92 references that are in the map. First posted on arXiv 24 May 2025.

Builds on

63 of 92 references.

References · 63

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  6. 2016Concrete Problems in AI Safetycited by 1,642
  7. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  8. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  9. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  10. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  11. 2023A Watermark for Large Language Modelscited by 725
  12. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
38 more
  1. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
  2. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  3. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
  4. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  5. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
  6. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
  7. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 387
  8. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  9. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 364
  10. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
  11. 2024Frontier Models are Capable of In-context Schemingcited by 287
  12. 2023Poisoning Language Models During Instruction Tuningcited by 269
  13. 2024Rethinking Machine Unlearning for Large Language Modelscited by 257
  14. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 223
  15. 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agentscited by 223
  16. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 194
  17. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 186
  18. 2023Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injectioncited by 157
  19. 2024Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversationscited by 129
  20. 2024PLeak: Prompt Leaking Attacks against Large Language Model Applicationscited by 127
  21. 2025Red-Teaming LLM Multi-Agent Systems via Communication Attackscited by 122
  22. 2023On the Exploitability of Instruction Tuningcited by 116
  23. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 107
  24. 2024Neural Exec: Learning (and Learning from) Execution Triggers for Prompt Injection Attackscited by 85
  25. 2023BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPTcited by 76
  26. 2025Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?cited by 75
  27. 2024Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedbackcited by 66
  28. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 65
  29. 2024Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applicationscited by 59
  30. 2024A Closer Look at Machine Unlearning for Large Language Modelscited by 56
  31. 2023Hijacking Large Language Models via Adversarial In-Context Learningcited by 45
  32. 2025Fully Autonomous AI Agents Should Not be Developedcited by 45
  33. 2024RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agentcited by 44
  34. 2025AutoRedTeamer: Autonomous Red Teaming with Lifelong Attack Integrationcited by 43
  35. 2023The Philosopher's Stone: Trojaning Plugins of Large Language Modelscited by 37
  36. 2025Defending against Indirect Prompt Injection by Instruction Detectioncited by 32
  37. 2024Embedding-based classifiers can detect prompt injection attackscited by 27
  38. 2025Nuclear Deployed: Analyzing Catastrophic Risks in Decision-making of Autonomous LLM Agentscited by 27

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 21.

29 papers.

Sort:

By area:

  1. 2025Enforcing Temporal Constraints for LLM Agentscited by 34
  2. 2026Security Threat Modeling for Emerging AI-Agent Protocols: A Comparative Analysis of MCP, A2A, Agora, and ANPcited by 15
  3. 2025KV Cache Transform Coding for Compact Storage in LLM Inferencecited by 13
  4. 2025A Benchmark for Evaluating Outcome-Driven Constraint Violations in Autonomous AI Agentscited by 13
  5. 2026SafeNeuron: Neuron-Level Safety Alignment for Large Language Modelscited by 10
  6. 2026Safe Language Generation in the Limitcited by 9
  7. 2025Bridging AI and Software Security: A Comparative Vulnerability Assessment of LLM Agent Deployment Paradigmscited by 7
  8. 2025Large Language Model Sourcing: A Surveycited by 5
  9. 2025Eliciting and Analyzing Emergent Misalignment in State-of-the-Art Large Language Modelscited by 4
  10. 2026Tone Matters: The Impact of Linguistic Tone on Hallucination in VLMscited by 4
  11. 2026Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Surveycited by 4
  12. 2025Trust Me, I Know This Function: Hijacking LLM Static Analysis using Biascited by 3
17 more
  1. 2026LLM-as-Judge Framework for Evaluating Tone-Induced Hallucination in Vision-Language Modelscited by 3
  2. 2025Countermind: A Multi-Layered Security Architecture for Large Language Modelscited by 2
  3. 2026Zero-Shot Embedding Drift Detection: A Lightweight Defense Against Prompt Injections in LLMscited by 2
  4. 2026The System Prompt Is the Attack Surface: How LLM Agent Configuration Shapes Security and Creates Exploitable Vulnerabilitiescited by 2
  5. 2026Accelerating Scientific Research with Gemini in the Real-Worldcited by 2
  6. 2026Layered LLM Defenses as an Ensemble: Access Tiers, Inference Cost, and the Measured Failure Correlation Between Defense Layerscited by 2
  7. 2026Gecko: A Simulation Environment with Stateful Feedback for Refining Agent Tool Callscited by 1
  8. 2026Stochasticity in Tokenisation Improves Robustnesscited by 1
  9. 2026Stealthy Backdoor Attacks against LLMs Based on Natural Style Triggerscited by 1
  10. 2026The Curse of Helpfulness: Inverse Scaling Law in Robustness to Distractor Instructions via DistractionIFcited by 1
  11. 2026Small, Free, and Effective: Orchestrating Open-Weight Small Language Models to Outperform Single LLM for Malware Analysiscited by 1
  12. 2026Monotonicity as an Architectural Bias for Robust Language Modelscited by 0
  13. 2026Integration of Object Detection and Small VLMs for Construction Safety Hazard Identificationcited by 0
  14. 2026On the Privacy of LLMs: An Ablation Studycited by 0
  15. 2026APIOT: Autonomous Vulnerability Management Across Bare-Metal Industrial OT Networkscited by 0
  16. 2026On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Modelscited by 0
  17. 2026MLCommons Jailbreak Benchmark v1.0cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Unique Security and Privacy Threats of Large Language Models: A Comprehensive Survey
  2. 2025Attack and defense techniques in large language models: A survey and new perspectives
  3. 2024Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models
  4. 2024Recent Advances in Attack and Defense Approaches of Large Language Models
  5. 2025A Survey of Attacks on Large Language Models
  6. 2024Exploring Vulnerabilities and Protections in Large Language Models: A Survey
  7. 2025Data Security in Large Language Models: Risks, Defense, and Directions
  8. 2024An Early Categorization of Prompt Injection Attacks on Large Language Models