mlsanityData to 7 Oct 2026

Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models

arXiv 2403.04786 ↗First posted 3 Mar 2024cs.CR cs.CLcited by 42

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have become a cornerstone in the field of Natural Language Processing (NLP), offering transformative capabilities in understanding and generating human-like text. However, with their rising prominence, the security and vulnerability aspects of these models have garnered significant attention. This paper presents a comprehensive survey of the various forms of attacks targeting LLMs, discussing the nature and mechanisms of these attacks, their potential impacts, and current defense strategies. We delve into topics such as adversarial attacks that aim to manipulate model outputs, data poisoning that affects model training, and privacy concerns related to training data exploitation. The paper also explores the effectiveness of different attack methodologies, the resilience of LLMs against these attacks, and the implications for model integrity and user trust. By examining the latest research, we provide insights into the current landscape of LLM vulnerabilities and defense mechanisms. Our objective is to offer a nuanced understanding of LLM attacks, foster awareness within the AI community, and inspire robust solutions to mitigate these risks in future developments.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 40 of its 61 references that are in the map. First posted on arXiv 3 Mar 2024.

Builds on

40 of 61 references.

References · 40

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  6. 2016Concrete Problems in AI Safetycited by 1,582
  7. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  8. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  10. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  11. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  12. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
28 more
  1. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
  2. 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
  3. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  4. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  5. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  6. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  7. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  8. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  9. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
  10. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  11. 2023Low-Resource Languages Jailbreak GPT-4cited by 290
  12. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  13. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 226
  14. 2023An Overview of Catastrophic AI Riskscited by 214
  15. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 202
  16. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  17. 2023Tensor Trust: Interpretable Prompt Injection Attacks from an Online Gamecited by 135
  18. 2023Explore, Establish, Exploit: Red Teaming Language Models from Scratchcited by 111
  19. 2023An LLM can Fool Itself: A Prompt-Based Adversarial Attackcited by 111
  20. 2023On the Exploitability of Instruction Tuningcited by 110
  21. 2023Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Modelscited by 87
  22. 2023Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competitioncited by 65
  23. 2022Adversarial Training for High-Stakes Reliabilitycited by 60
  24. 2023LLM Censorship: A Machine Learning Challenge or a Computer Security Problem?cited by 47
  25. 2023The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Riskscited by 35
  26. 2023Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Informationcited by 32
  27. 2023Prompt Packer: Deceiving LLMs through Compositional Instruction with Hidden Attackscited by 30
  28. 2023Robust Safety Classifier for Large Language Models: Adversarial Prompt Shieldcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 16; 2025: 24; 2026: 2.

42 papers.

Sort:

By area:

  1. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  2. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  3. 2025Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attackscited by 49
  4. 2025Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiescited by 36
  5. 2024Tokenization Matters! Degrading Large Language Models through Challenging Their Tokenizationcited by 32
  6. 2025Accumulating Context Changes the Beliefs of Language Modelscited by 25
  7. 2024What Features in Prompts Jailbreak LLMs? Investigating the Mechanisms Behind Attackscited by 24
  8. 2024HiddenGuard: Fine-Grained Safe Generation with Specialized Representation Routercited by 16
  9. 2024Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformerscited by 14
  10. 2025Survey of Adversarial Robustness in Multimodal Large Language Modelscited by 14
  11. 2024A Large-Scale Exploit Instrumentation Study of AI/ML Supply Chain Attacks in Hugging Face Modelscited by 12
  12. 2025From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystemcited by 11
30 more
  1. 2025Single-pass Detection of Jailbreaking Input in Large Language Modelscited by 10
  2. 2024HarmLevelBench: Evaluating Harm-Level Compliance and the Impact of Quantization on Model Alignmentcited by 9
  3. 2025Probing Latent Subspaces in LLM for AI Security: Identifying and Manipulating Adversarial Statescited by 8
  4. 2025PiCo: Jailbreaking Multimodal Large Language Models via Pictorial Code Contextualizationcited by 8
  5. 2024LLM-Virus: Evolutionary Jailbreak Attack on Large Language Modelscited by 7
  6. 2024PII-Scope: A Comprehensive Study on Training Data PII Extraction Attacks in LLMscited by 6
  7. 2025RedTeamLLM: an Agentic AI framework for offensive securitycited by 6
  8. 2024Recent Advances in Attack and Defense Approaches of Large Language Modelscited by 5
  9. 2025Large Language Models Are Unreliable for Cyber Threat Intelligencecited by 5
  10. 2024On the Ethical Considerations of Generative Agentscited by 4
  11. 2025On the Robustness of Transformers against Context Hijacking for Linear Classificationcited by 4
  12. 2025Giving AI Agents Access to Cryptocurrency and Smart Contracts Creates New Vectors of AI Harmcited by 3
  13. 2025Thought Purity: A Defense Framework For Chain-of-Thought Attackcited by 3
  14. 2024Large Language Models as Carriers of Hidden Messagescited by 2
  15. 2024Too Big to Fool: Resisting Deception in Language Modelscited by 2
  16. 2025Prompt Inject Detection with Generative Explanation as an Investigative Toolcited by 2
  17. 2025Helping Large Language Models Protect Themselves: An Enhanced Filtering and Summarization Systemcited by 2
  18. 2025LoRA Users Beware: A Few Spurious Tokens Can Manipulate Your Finetuned Modelcited by 2
  19. 2025Securing Educational LLMs: A Generalised Taxonomy of Attacks on LLMs and DREAD Risk Assessmentcited by 2
  20. 2025ERIS: Evolutionary Real-world Interference Scheme for Jailbreaking Audio Large Modelscited by 2
  21. 2024Hallucinating AI Hijacking Attack: Large Language Models and Malicious Code Recommenderscited by 1
  22. 2025Linearly Decoding Refused Knowledge in Aligned Language Modelscited by 1
  23. 2025Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generationcited by 1
  24. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  25. 2025VisualDAN: Exposing Vulnerabilities in VLMs with Visual-Driven DAN Commandscited by 1
  26. 2024A Simple Architecture for Enterprise Large Language Model Applications based on Role based security and Clearance Levels using Retrieval-Augmented Generation or Mixture of Expertscited by 0
  27. 2025Preventing Adversarial AI Attacks Against Autonomous Situational Awareness: A Maritime Case Studycited by 0
  28. 2025Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense Techniquescited by 0
  29. 2026Reasoning-targeted Jailbreak Attacks on Large Reasoning Models via Semantic Triggers and Psychological Framingcited by 0
  30. 2026Truth Lies Deep: Countering Semantic Camouflage via Latent Intent Verificationcited by 0