mlsanityData to 7 Oct 2026

Recent Advances in Attack and Defense Approaches of Large Language Models

arXiv 2409.03274 ↗First posted 5 Sep 2024cs.CR cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have revolutionized artificial intelligence and machine learning through their advanced text processing and generating capabilities. However, their widespread deployment has raised significant safety and reliability concerns. Established vulnerabilities in deep neural networks, coupled with emerging threat models, may compromise security evaluations and create a false sense of security. Given the extensive research in the field of LLM security, we believe that summarizing the current state of affairs will help the research community better understand the present landscape and inform future developments. This paper reviews current research on LLM vulnerabilities and threats, and evaluates the effectiveness of contemporary defense mechanisms. We analyze recent studies on attack vectors and model weaknesses, providing insights into attack mechanisms and the evolving threat landscape. We also examine current defense strategies, highlighting their strengths and limitations. By contrasting advancements in attack and defense methodologies, we identify research gaps and propose future directions to enhance LLM security. Our goal is to advance the understanding of LLM safety challenges and guide the development of more robust security measures.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 156 of its 190 references that are in the map. First posted on arXiv 5 Sep 2024.

Builds on

156 of 190 references.

References · 156

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2013Intriguing properties of neural networkscited by 5,114
  9. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  10. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  11. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  12. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
38 more
  1. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  3. 2020Extracting Training Data from Large Language Modelscited by 2,144
  4. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  5. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  6. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  7. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  8. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  10. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  11. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  12. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  13. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  14. 2022Red Teaming Language Models with Language Modelscited by 983
  15. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  16. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  17. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  18. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
  19. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  20. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  21. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  22. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  23. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  24. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 679
  25. 2023Instruction Tuning with GPT-4cited by 676
  26. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  27. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  28. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  29. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  30. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  31. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  32. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  33. 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 490
  34. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 484
  35. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  36. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  37. 2022Memory-Based Model Editing at Scalecited by 443
  38. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439

50 most cited of 156 All 156 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2025MedSentry: Understanding and Mitigating Safety Risks in Medical LLM Multi-Agent Systemscited by 8
  2. 2025SocialHarmBench: Revealing LLM Vulnerabilities to Socially Harmful Requestscited by 5
  3. 2026DRAINCODE: Stealthy Energy Consumption Attacks on Retrieval-Augmented Code Generation via Context Poisoningcited by 3
  4. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  5. 2025TrajSyn: Privacy-Preserving Dataset Distillation from Federated Model Trajectories for Server-Side Adversarial Trainingcited by 0