mlsanityData to 14 Oct 2026

Exploring Vulnerabilities and Protections in Large Language Models: A Survey

arXiv 2406.00240 ↗First posted 1 Jun 2024cs.LG cs.CL cs.CRcited by 11

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

As Large Language Models (LLMs) increasingly become key components in various AI applications, understanding their security vulnerabilities and the effectiveness of defense mechanisms is crucial. This survey examines the security challenges of LLMs, focusing on two main areas: Prompt Hacking and Adversarial Attacks, each with specific types of threats. Under Prompt Hacking, we explore Prompt Injection and Jailbreaking Attacks, discussing how they work, their potential impacts, and ways to mitigate them. Similarly, we analyze Adversarial Attacks, breaking them down into Data Poisoning Attacks and Backdoor Attacks. This structured examination helps us understand the relationships between these vulnerabilities and the defense strategies that can be implemented. The survey highlights these security challenges and discusses robust defensive frameworks to protect LLMs against these threats. By detailing these security issues, the survey contributes to the broader discussion on creating resilient AI systems that can resist sophisticated attacks.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 31 of its 42 references that are in the map. First posted on arXiv 1 Jun 2024.

Builds on

31 of 42 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Security and Privacy Challenges of Large Language Models: A Survey30 Jan
  2. 2023Prompt Packer: Deceiving LLMs through Compositional Instruction with Hidden Attacks16 Oct

Other references · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  4. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  5. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 916
  6. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  7. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
  8. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 498
  9. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 469
  10. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 356
  11. 2023Poisoning Language Models During Instruction Tuningcited by 269
  12. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 263
17 more
  1. 2022Demystifying Prompts in Language Models via Perplexity Estimationcited by 216
  2. 2023Open Sesame! Universal Black Box Jailbreaking of Large Language Modelscited by 134
  3. 2023ProPILE: Probing Privacy Leakage in Large Language Modelscited by 129
  4. 2021Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoningcited by 113
  5. 2021RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Modelscited by 104
  6. 2023Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Modelscited by 94
  7. 2020Trojaning Language Models for Fun and Profitcited by 90
  8. 2022Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methodscited by 79
  9. 2022A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarkscited by 74
  10. 2023Composite Backdoor Attacks Against Large Language Modelscited by 73
  11. 2023Adversarial Demonstration Attacks on Large Language Modelscited by 63
  12. 2023TrojanPuzzle: Covertly Poisoning Code-Suggestion Modelscited by 60
  13. 2022Fine-mixing: Mitigating Backdoors in Fine-tuned Language Modelscited by 57
  14. 2023A Comprehensive Overview of Backdoor Attacks in Large Language Models within Communication Networkscited by 42
  15. 2023Backdoor Attacks and Defenses in Federated Learning: Survey, Challenges and Future Research Directionscited by 33
  16. 2023Defending Pre-trained Language Models as Few-shot Learners against Backdoor Attackscited by 33
  17. 2023AI Risk Skepticism, A Comprehensive Surveycited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 6; 2026: 1.

11 papers.

  1. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 26
  2. 2025When Models Reason in Your Language: Controlling Thinking Language Comes at the Cost of Accuracycited by 14
  3. 2025LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasurescited by 8
  4. 2025Adversarial Attacks on LLM-as-a-Judge Systems: Insights from Prompt Injectionscited by 6
  5. 2025POISONCRAFT: Practical Poisoning of Retrieval-Augmented Generation for Large Language Modelscited by 6
  6. 2024eFedLLM: Efficient LLM Inference Based on Federated Learningcited by 3
  7. 2024On the Validity of Traditional Vulnerability Scoring Systems for Adversarial Attacks against LLMscited by 2
  8. 2026Layered LLM Defenses as an Ensemble: Access Tiers, Inference Cost, and the Measured Failure Correlation Between Defense Layerscited by 2
  9. 2024Hallucinating AI Hijacking Attack: Large Language Models and Malicious Code Recommenderscited by 1
  10. 2024Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyondcited by 1
  11. 2025Enhancing Social Robots through Resilient AIcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Models
  2. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks
  3. 2025Attack and defense techniques in large language models: A survey and new perspectives
  4. 2024Recent Advances in Attack and Defense Approaches of Large Language Models
  5. 2024Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models
  6. 2024SoK: Prompt Hacking of Large Language Models
  7. 2025Breaking to Build: A Threat Model of Prompt-Based Attacks for Securing LLMs
  8. 2025Data Security in Large Language Models: Risks, Defense, and Directions