mlsanityData to 14 Oct 2026

Attack and defense techniques in large language models: A survey and new perspectives

arXiv 2505.00976 ↗First posted 2 May 2025cs.CR cs.AI cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have become central to numerous natural language processing tasks, but their vulnerabilities present significant security and ethical challenges. This systematic survey explores the evolving landscape of attack and defense techniques in LLMs. We classify attacks into adversarial prompt attack, optimized attacks, model theft, as well as attacks on application of LLMs, detailing their mechanisms and implications. Consequently, we analyze defense strategies, including prevention-based and detection-based defense methods. Although advances have been made, challenges remain to adapt to the dynamic threat landscape, balance usability with robustness, and address resource constraints in defense implementation. We highlight open problems, including the need for adaptive scalable defenses, explainable security techniques, and standardized evaluation frameworks. This survey provides actionable insights and directions for developing secure and resilient LLMs, emphasizing the importance of interdisciplinary collaboration and ethical considerations to mitigate risks in real-world applications.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 60 of its 88 references that are in the map. First posted on arXiv 2 May 2025.

Builds on

60 of 88 references.

References · 60

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  4. 2020Extracting Training Data from Large Language Modelscited by 2,189
  5. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  6. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  7. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  8. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  9. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  10. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  11. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  12. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
38 more
  1. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  2. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
  3. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 498
  4. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 469
  5. 2023Detecting Language Model Attacks with Perplexitycited by 431
  6. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 367
  7. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 364
  8. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 278
  9. 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 200
  10. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 194
  11. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 186
  12. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbotscited by 183
  13. 2023Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Modelscited by 154
  14. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 145
  15. 2024Optimization-based Prompt Injection Attack to LLM-as-a-Judgecited by 143
  16. 2019Thieves on Sesame Street! Model Extraction of BERT-based APIscited by 139
  17. 2024GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysiscited by 96
  18. 2024Attention Tracker: Detecting Prompt Injection Attacks in LLMscited by 95
  19. 2023Protecting Language Generation Models via Invisible Watermarkingcited by 90
  20. 2023MedViT: A Robust Vision Transformer for Generalized Medical Image Classificationcited by 89
  21. 2023Practical and Ethical Challenges of Large Language Models in Education: A Systematic Scoping Reviewcited by 83
  22. 2020Imitation Attacks and Defenses for Black-box Machine Translation Systemscited by 81
  23. 2022CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarkscited by 76
  24. 2023Attack Prompt Generation for Red Teaming and Defending Large Language Modelscited by 76
  25. 2023Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Modelscited by 75
  26. 2024Parameter-Efficient Fine-Tuning with Discrete Fourier Transformcited by 71
  27. 2022Evaluating the Susceptibility of Pre-Trained Language Models via Handcrafted Adversarial Examplescited by 69
  28. 2023Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermarkcited by 67
  29. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
  30. 2023Jailbreaking GPT-4V via Self-Adversarial Attacks with System Promptscited by 65
  31. 2023Adversarial Demonstration Attacks on Large Language Modelscited by 63
  32. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Mannercited by 63
  33. 2023Assessing Prompt Injection Risks in 200+ Custom GPTscited by 55
  34. 2024Signed-Prompt: A New Approach to Prevent Prompt Injection Attacks Against LLM-Integrated Applicationscited by 54
  35. 2022Fairness-aware Adversarial Perturbation Towards Bias Mitigation for Deployed Deep Modelscited by 47
  36. 2024Distract Large Language Models for Automatic Jailbreak Attackcited by 45
  37. 2024The Ethics of ChatGPT in Medicine and Healthcare: A Systematic Review on Large Language Models (LLMs)cited by 45
  38. 2023Large Language Models for Telecom: Forthcoming Impact on the Industrycited by 44

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 12.

14 papers.

Sort:

By area:

  1. 2026From Spark to Fire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaborationcited by 14
  2. 2026ADAM: A Systematic Data Extraction Attack on Agent Memory via Adaptive Queryingcited by 5
  3. 2026Enforcing Benign Trajectories: A Behavioral Firewall for Structured-Workflow AI Agentscited by 3
  4. 2025Shaking to Reveal: Perturbation-Based Detection of LLM Hallucinationscited by 2
  5. 2026Proactive Hardening of LLM Defenses with HASTEcited by 2
  6. 2025Sentra-Guard: A Real-Time Multilingual Defense Against Adversarial LLM Promptscited by 1
  7. 2026JailbreakOPT: Tool-Assisted Iterative Jailbreak Prompt Optimizationcited by 1
  8. 2026When Context Bites: Detecting RAG Poisoning via Document-Level Attention Collapsecited by 1
  9. 2026Security Assessment and Mitigation Strategies for Large Language Models: A Comprehensive Defensive Frameworkcited by 0
  10. 2026Dictionary-Aligned Concept Control for Safeguarding Multimodal LLMscited by 0
  11. 2026Conversations Risk Detection LLMs in Financial Agents via Multi-Stage Generative Rolloutcited by 0
  12. 2026Reasoning-targeted Jailbreak Attacks on Large Reasoning Models via Semantic Triggers and Psychological Framingcited by 0
2 more
  1. 2026AVISE: Framework for Evaluating the Security of AI Systemscited by 0
  2. 2026Beyond ℓ2-norm and ℓ_∞-norm: A Curvature-Inspired ℓp-Norm Scheme for Deep Neural Networkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models
  2. 2024Recent Advances in Attack and Defense Approaches of Large Language Models
  3. 2024Exploring Vulnerabilities and Protections in Large Language Models: A Survey
  4. 2025Security Concerns for Large Language Models: A Survey
  5. 2025Data Security in Large Language Models: Risks, Defense, and Directions
  6. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Models
  7. 2025A Survey on Backdoor Threats in Large Language Models (LLMs): Attacks, Defenses, and Evaluations
  8. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey