mlsanityData to 7 Oct 2026

Break the Breakout: Reinventing LM Defense Against Jailbreak Attacks with Self-Refinement

arXiv 2402.15180 ↗First posted 23 Feb 2024cs.LG cs.CL cs.CRcited by 25

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Caution: This paper includes offensive words that could potentially cause unpleasantness. Language models (LMs) are vulnerable to exploitation for adversarial misuse. Training LMs for safety alignment is extensive and makes it hard to respond to fast-developing attacks immediately, such as jailbreaks. We propose self-refine with formatting that achieves outstanding safety even in non-safety-aligned LMs and evaluate our method alongside several defense baselines, demonstrating that it is the safest training-free method against jailbreak attacks. Additionally, we proposed a formatting method that improves the efficiency of the self-refine process while reducing attack success rates in fewer iterations. We've also observed that non-safety-aligned LMs outperform safety-aligned LMs in safety tasks by giving more helpful and safe responses. In conclusion, our findings can achieve less safety risk with fewer computational costs, allowing non-safety LM to be easily utilized in real-world service.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 19 of its 26 references that are in the map. First posted on arXiv 23 Feb 2024.

Builds on

19 of 26 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  2. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  4. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul
  5. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study23 May

Other references · 14

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  9. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  10. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  11. 2023Zephyr: Direct Distillation of LM Alignmentcited by 498
  12. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 424
2 more
  1. 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 250
  2. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 213

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 9; 2026: 5.

25 papers.

Sort:

By area:

  1. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  2. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  3. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothingcited by 68
  4. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  5. 2024BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attackscited by 28
  6. 2024From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreakingcited by 22
  7. 2024Mission Impossible: A Statistical Perspective on Jailbreaking LLMscited by 21
  8. 2025Monitoring Decomposition Attacks in LLMs with Lightweight Sequential Monitorscited by 21
  9. 2024PAPILLON: Efficient and Stealthy Fuzz Testing-Powered Jailbreaks for LLMscited by 20
  10. 2025Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study of Conditional Effectivenesscited by 18
  11. 2024Intrinsic Self-correction for Enhanced Morality: An Analysis of Internal Mechanisms and the Superficial Hypothesiscited by 15
  12. 2025Single-pass Detection of Jailbreaking Input in Large Language Modelscited by 10
13 more
  1. 2025Proactive defense against LLM Jailbreakcited by 6
  2. 2025Secure Tug-of-War (SecTOW): Iterative Defense-Attack Training with Reinforcement Learning for Multimodal Model Securitycited by 4
  3. 2025SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defensescited by 4
  4. 2024Preventing Jailbreak Prompts as Malicious Tools for Cybercriminals: A Cyber Defense Perspectivecited by 3
  5. 2024PrisonBreak: Jailbreaking Large Language Models with at Most Twenty-Five Targeted Bit-flipscited by 3
  6. 2025Can Small Language Models Reliably Resist Jailbreak Attacks? A Comprehensive Evaluationcited by 3
  7. 2026Fail-Closed Alignment for Large Language Modelscited by 2
  8. 2025Enhancing the Safety of Medical Vision-Language Models by Synthetic Demonstrationscited by 1
  9. 2026LoopTrap: Termination Poisoning Attacks on LLM Agentscited by 1
  10. 2026Dynamic Deep Prompt Optimization for Defending Against Jailbreak Attacks on LLMscited by 1
  11. 2025SoK: a Comprehensive Causality Analysis Framework for Large Language Model Securitycited by 0
  12. 2026A Systematic Study of Training-Free Methods for Trustworthy Large Language Modelscited by 0
  13. 2026From Interpretability to Control: Insights from Six Years of the TrustNLP Workshopcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing
  2. 2025Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers
  3. 2025Forewarned is Forearmed: Pre-Synthesizing Jailbreak-like Instructions to Enhance LLM Safety Guardrail to Potential Attacks
  4. 2026Break Me If You Can: Self-Jailbreaking of Aligned LLMs via Lexical Insertion Prompting
  5. 2025Jailbreaking Attacks vs. Content Safety Filters: How Far Are We in the LLM Safety Arms Race?
  6. 2025SafeLLM: Unlearning Harmful Outputs from Large Language Models against Jailbreak Attacks
  7. 2025SecurityLingua: Efficient Defense of LLM Jailbreak Attacks via Security-Aware Prompt Compression
  8. 2026MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety