mlsanityData to 14 Oct 2026

The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensiveness

arXiv 2401.00287 ↗First posted 30 Dec 2023cs.CLcited by 67

Not on the map. Why?

Cited by 38 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

As Large Language Models (LLMs) play an increasingly pivotal role in natural language processing applications, their safety concerns become critical areas of NLP research. This paper presents Safety and Over-Defensiveness Evaluation (SODE) benchmark: a collection of diverse safe and unsafe prompts with carefully designed evaluation methods that facilitate systematic evaluation, comparison, and analysis over 'safety' and 'over-defensiveness.' With SODE, we study a variety of LLM defense strategies over multiple state-of-the-art LLMs, which reveals several interesting and important findings, such as (a) the widely popular 'self-checking' techniques indeed improve the safety against unsafe inputs, but this comes at the cost of extreme over-defensiveness on the safe inputs, (b) providing a safety instruction along with in-context exemplars (of both safe and unsafe inputs) consistently improves safety and also mitigates undue over-defensiveness of the models, (c) providing contextual knowledge easily breaks the safety guardrails and makes the models more vulnerable to generating unsafe responses. Overall, our work reveals numerous such critical findings that we believe will pave the way and facilitate further research in improving the safety of LLMs.

Key facts. Cited by 67 papers in the map (data through 14 Oct 2026). Builds on 35 of its 56 references that are in the map. First posted on arXiv 30 Dec 2023.

Builds on

35 of 56 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  8. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  9. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  10. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  11. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  12. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
23 more
  1. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 559
  2. 2023Detecting Language Model Attacks with Perplexitycited by 431
  3. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 431
  4. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
  5. 2023Certifying LLM Safety against Adversarial Promptingcited by 266
  6. 2022Prompting GPT-3 To Be Reliablecited by 260
  7. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 244
  8. 2020Selective Question Answering under Domain Shiftcited by 238
  9. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 223
  10. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 206
  11. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 192
  12. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 184
  13. 2023Attack Prompt Generation for Red Teaming and Defending Large Language Modelscited by 76
  14. 2022Investigating Selective Prediction Approaches Across Several Tasks in IID, OOD, and Adversarial Settingscited by 66
  15. 2023Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Modelscited by 60
  16. 2023Bypassing the Safety Training of Open-Source LLMs with Priming Attackscited by 50
  17. 2023How Trustworthy are Open-Source LLMs? An Assessment under Malicious Demonstrations Shows their Vulnerabilitiescited by 33
  18. 2023Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Informationcited by 33
  19. 2020Towards Improving Selective Prediction Ability of NLP Systemscited by 20
  20. 2023Co2PT: Mitigating Bias in Pre-trained Language Models through Counterfactual Contrastive Prompt Tuningcited by 15
  21. 2023Safety Alignment in NLP Tasks: Weakly Aligned Summarization as an In-Context Attackcited by 11
  22. 2022Can Open-Domain QA Reader Utilize External Knowledge Efficiently like Humans?cited by 10
  23. 2023Learn What NOT to Learn: Towards Generative Safety in Chatbotscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 26; 2025: 30; 2026: 11.

67 papers.

Sort:

By area:

  1. 2024AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Expertscited by 136
  2. 2025Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrailscited by 132
  3. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 121
  4. 2024Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapescited by 74
  5. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 70
  6. 2024Large Language Model Safety: A Holistic Surveycited by 59
  7. 2025From Hard Refusals to Safe-Completions: Toward Output-Centric Safety Trainingcited by 52
  8. 2024SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steeringcited by 50
  9. 2024A Theoretical Understanding of Self-Correction through In-context Alignmentcited by 44
  10. 2024Boosting Jailbreak Attack with Momentumcited by 40
  11. 2024Knowledge Boundary of Large Language Models: A Surveycited by 40
  12. 2024Automatic Pseudo-Harmful Prompt Generation for Evaluating False Refusals in Large Language Modelscited by 35
38 more
  1. 2024ASETF: A Novel Method for Jailbreak Attack on LLMs through Translate Suffix Embeddingscited by 32
  2. 2024Defending Jailbreak Prompts via In-Context Adversarial Gamecited by 29
  3. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 26
  4. 2024Prompt Leakage effect and defense strategies for multi-turn LLM interactionscited by 23
  5. 2025X-Boundary: Establishing Exact Safety Boundary to Shield LLMs from Multi-Turn Jailbreaks without Compromising Usabilitycited by 20
  6. 2025Cancer-Myth: Evaluating Large Language Models on Patient Questions with False Presuppositionscited by 15
  7. 2025Epistemic Alignment: A Mediating Framework for User-LLM Knowledge Deliverycited by 14
  8. 2025SecurityLingua: Efficient Defense of LLM Jailbreak Attacks via Security-Aware Prompt Compressioncited by 13
  9. 2025Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metricscited by 12
  10. 2025From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystemcited by 12
  11. 2025GRAIT: Gradient-Driven Refusal-Aware Instruction Tuning for Effective Hallucination Mitigationcited by 11
  12. 2025Think Twice, Generate Once: Safeguarding by Progressive Self-Reflectioncited by 11
  13. 2024Chaos with Keywords: Exposing Large Language Models Sycophantic Hallucination to Misleading Keywords and Evaluating Defense Strategiescited by 10
  14. 2024PrimeGuard: Safe and Helpful LLMs through Tuning-Free Routingcited by 10
  15. 2025Beyond Surface-Level Patterns: An Essence-Driven Defense Framework Against Jailbreak Attacks in LLMscited by 9
  16. 2024When to Speak, When to Abstain: Contrastive Decoding with Abstentioncited by 8
  17. 2025PiCo: Jailbreaking Multimodal Large Language Models via Pictorial Code Contextualizationcited by 8
  18. 2025LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasurescited by 8
  19. 2024LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Modelscited by 7
  20. 2024SoK: Prompt Hacking of Large Language Modelscited by 7
  21. 2024Token Highlighter: Inspecting and Mitigating Jailbreak Prompts for Large Language Modelscited by 7
  22. 2025The TIP of the Iceberg: Revealing a Hidden Class of Task-in-Prompt Adversarial Attacks on LLMscited by 7
  23. 2025Retrieval-Augmented Purifier for Robust LLM-Empowered Recommendationcited by 7
  24. 2026Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignmentcited by 7
  25. 2025Code Red! On the Harmfulness of Applying Off-the-shelf Large Language Models to Programming Taskscited by 6
  26. 2025Exploring Backdoor Attack and Defense for LLM-empowered Recommendationscited by 6
  27. 2024Alignment with Preference Optimization Is All You Need for LLM Safetycited by 5
  28. 2024The Tug of War Within: Mitigating the Fairness-Privacy Conflicts in Large Language Modelscited by 5
  29. 2025Adversarial Suffix Filtering: a Defense Pipeline for LLMscited by 5
  30. 2025UniErase: Towards Balanced and Precise Unlearning in Language Modelscited by 5
  31. 2026Evolving Contextual Safety in Multi-Modal Large Language Models via Inference-Time Self-Reflective Memorycited by 5
  32. 2024FATS: A Prompt Injection Attack Utilizing Feign Security Agents with Deceptive Few-shots Learningcited by 4
  33. 2025SafeInt: Shielding Large Language Models from Jailbreak Attacks via Safety-Aware Representation Interventioncited by 4
  34. 2025NeuronTune: Fine-Grained Neuron Modulation for Balanced Safety-Utility Alignment in LLMscited by 4
  35. 2025DialogGuard: Multi-Agent Psychosocial Safety Evaluation of Sensitive LLM Responsescited by 3
  36. 2024No Free Lunch for Defending Against Prefilling Attack by In-Context Learningcited by 2
  37. 2025The Structural Safety Generalization Problemcited by 2
  38. 2024ASPIRER: Bypassing System Prompts With Permutation-based Backdoors in LLMscited by 1

50 most cited of 67 All 67 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Attack and defense techniques in large language models: A survey and new perspectives
  2. 2025LLM-Safety Evaluations Lack Robustness
  3. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models
  4. 2024S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Models
  5. 2026Trust The Typical
  6. 2024Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models
  7. 2025Are Smarter LLMs Safer? Exploring Safety-Reasoning Trade-offs in Prompting and Fine-Tuning
  8. 2024ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stages