mlsanityData to 7 Oct 2026

LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked

arXiv 2308.07308 ↗First posted 14 Aug 2023cs.CL cs.AIcited by 226

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

Large language models (LLMs) are popular for high-quality text generation but can produce harmful content, even when aligned with human values through reinforcement learning. Adversarial prompts can bypass their safety measures. We propose LLM Self Defense, a simple approach to defend against these attacks by having an LLM screen the induced responses. Our method does not require any fine-tuning, input preprocessing, or iterative output generation. Instead, we incorporate the generated content into a pre-defined prompt and employ another instance of an LLM to analyze the text and predict whether it is harmful. We test LLM Self Defense on GPT 3.5 and Llama 2, two of the current most prominent LLMs against various types of attacks, such as forcefully inducing affirmative responses to prompts and prompt engineering attacks. Notably, LLM Self Defense succeeds in reducing the attack success rate to virtually 0 using both GPT 3.5 and Llama 2. The code is publicly available at https://github.com/poloclub/llm-self-defense

Key facts. Cited by 226 papers in the map (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 29 of its 42 references that are in the map. First posted on arXiv 14 Aug 2023.

Builds on

29 of 42 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy3 Jul
  2. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection23 Feb

Other references · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  10. 2023A Survey of Large Language Modelscited by 2,537
  11. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  12. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
15 more
  1. 2022A Survey on In-context Learningcited by 717
  2. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  3. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  4. 2023MusicLM: Generating Music From Textcited by 442
  5. 2017Adversarial Examples: Attacks and Defenses for Deep Learningcited by 350
  6. 2023Are aligned neural networks adversarially aligned?cited by 305
  7. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 304
  8. 2023AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Headcited by 264
  9. 2023Evaluating the Moral Beliefs Encoded in LLMscited by 211
  10. 2023SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoningcited by 157
  11. 2021Globally-Robust Neural Networkscited by 106
  12. 2021What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpuscited by 79
  13. 2023Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Modelscited by 56
  14. 2021The Threat of Offensive AI to Organizationscited by 37
  15. 2023Flows: Building Blocks of Reasoning and Collaborating AIcited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 16; 2024: 85; 2025: 91; 2026: 34.

226 papers.

Sort:

By area:

  1. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  2. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  3. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  4. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  5. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
  6. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  7. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
  8. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  9. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  10. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 173
  11. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 135
  12. 2024Attacks, Defenses and Evaluations for LLM Conversation Safety: A Surveycited by 124
38 more
  1. 2024Optimization-based Prompt Injection Attack to LLM-as-a-Judgecited by 123
  2. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  3. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  4. 2023AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Modelscited by 102
  5. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Modelscited by 95
  6. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 86
  7. 2023Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queriescited by 82
  8. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 80
  9. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editingcited by 77
  10. 2024Safeguarding Large Language Models: A Surveycited by 73
  11. 2024FlipAttack: Jailbreak LLMs via Flippingcited by 73
  12. 2024On the Role of Attention Heads in Large Language Model Safetycited by 70
  13. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothingcited by 68
  14. 2024Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defensescited by 68
  15. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  16. 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 67
  17. 2024Intention Analysis Makes LLMs A Good Jailbreak Defendercited by 64
  18. 2023The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensivenesscited by 63
  19. 2024Fight Back Against Jailbreaking via Prompt Adversarial Tuningcited by 63
  20. 2024Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypescited by 58
  21. 2024EasyJailbreak: A Unified Framework for Jailbreaking Large Language Modelscited by 57
  22. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Mannercited by 57
  23. 2025G-Safeguard: A Topology-Guided Security Lens and Treatment on LLM-based Multi-agent Systemscited by 56
  24. 2023Self-Guard: Empower the LLM to Safeguard Itselfcited by 52
  25. 2023Adversarial Attacks and Defenses in Large Language Models: Old and New Threatscited by 49
  26. 2024Risks of AI Scientists: Prioritizing Safeguarding Over Autonomycited by 49
  27. 2024PRP: Propagating Universal Perturbations to Attack Large Language Model Guard-Railscited by 49
  28. 2024Eraser: Jailbreaking Defense in Large Language Models via Unlearning Harmful Knowledgecited by 48
  29. 2024Confidence Matters: Revisiting Intrinsic Self-Correction Capabilities of Large Language Modelscited by 47
  30. 2023Can LLMs Follow Simple Rules?cited by 45
  31. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  32. 2024Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Modelscited by 42
  33. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 33
  34. 2025Towards Trustworthy GUI Agents: A Surveycited by 33
  35. 2025The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMscited by 32
  36. 2024ASETF: A Novel Method for Jailbreak Attack on LLMs through Translate Suffix Embeddingscited by 31
  37. 2023Bergeron: Combating Adversarial Attacks through a Conscience-Based Alignment Frameworkcited by 29
  38. 2024PARDEN, Can You Repeat That? Defending against Jailbreaks via Repetitioncited by 29

50 most cited of 226 All 226 (JSON)