mlsanityData to 14 Oct 2026

Jailbreak to Protect: Buffering and Reinforcing via Temporary Jailbreaking for Safe Fine-Tuning in Large Language Models

arXiv 2605.24550 ↗First posted 23 May 2026cs.AI cs.CL cs.LGcited by 0

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Fine-tuning-as-a-Service (FaaS) enables personalization of large language models (LLMs), but it can weaken safety-alignment under harmful fine-tuning attacks. Recent work has shown that activating harmful-behavior modules during fine-tuning can prevent models from learning undesired behaviors, but its mechanism remains unclear. In this paper, we revisit temporary jailbreaking as a defense against harmful fine-tuning and provide a gradient-level analysis showing that it saturates safety-degrading gradients while preserving benign task-relevant gradients. Based on this insight, we propose a Buffer-and-Reinforce fine-tuning framework that buffers harmful updates during user fine-tuning and reinforces safety after adaptation. Specifically, BufferLoRA induces temporary jailbreaking as a removable adapter to reduce harmful updates during user fine-tuning. After adaptation, ReinforceLoRA, trained to recover refusal behavior under the temporarily jailbroken state, is integrated with UserLoRA via QR decomposition-based merging to reinforce safety while preserving user-task performance. Extensive experiments show that our framework achieves superior safety and utility with no additional safety data during user fine-tuning and minimal computational cost.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 48 of its 58 references that are in the map. First posted on arXiv 23 May 2026.

Builds on

48 of 58 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs22 Jul
  2. 2024Efficient Adversarial Training in LLMs with Continuous Attacks24 May
  3. 2024Intention Analysis Makes LLMs A Good Jailbreak Defender12 Jan
  4. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 42

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  7. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  8. 2022Editing Models with Task Arithmeticcited by 1,120
  9. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  10. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  11. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 818
  12. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
30 more
  1. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
  2. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
  3. 2024Improving Alignment and Robustness with Circuit Breakerscited by 322
  4. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 261
  5. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 195
  6. 2025SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilitiescited by 136
  7. 2024Safety Layers in Aligned Large Language Models: The Key to LLM Securitycited by 134
  8. 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 125
  9. 2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 115
  10. 2023Fine-tuning can cripple your foundation model; preserving features may be the solutioncited by 96
  11. 2024Representation Noising: A Defence Mechanism Against Harmful Finetuningcited by 96
  12. 2024Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 92
  13. 2024Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbationcited by 92
  14. 2025LLMs Encode Harmfulness and Refusal Separatelycited by 88
  15. 2024Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuningcited by 72
  16. 2024Robust LLM safeguarding via refusal feature adversarial trainingcited by 64
  17. 2024Merging LoRAs like Playing LEGO: Pushing the Modularity of LoRA to Extremes Through Rank-Wise Clusteringcited by 59
  18. 2024R2-Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge-Enhanced Logical Reasoningcited by 43
  19. 2024Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbationcited by 37
  20. 2024NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuningcited by 37
  21. 2025SafeSwitch: Steering Unsafe LLM Behavior via Internal Activation Signalscited by 29
  22. 2025SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignmentcited by 24
  23. 2023Making Harmful Behaviors Unlearnable for Large Language Modelscited by 20
  24. 2025Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbationcited by 20
  25. 2025AsFT: Anchoring Safety During LLM Fine-Tuning Within Narrow Safety Basincited by 19
  26. 2025Safe Delta: Consistently Preserving Safety when Fine-Tuning LLMs on Diverse Datasetscited by 18
  27. 2025Vulnerability-Aware Alignment: Mitigating Uneven Forgetting in Harmful Fine-Tuningcited by 17
  28. 2025LoX: Low-Rank Extrapolation Robustifies LLM Safety Against Fine-tuningcited by 12
  29. 2025Pharmacist: Safety Alignment Data Curation for Large Language Models against Harmful Fine-tuningcited by 6
  30. 2025Safety-Aligned Weights Are Not Enough: Refusal-Teacher-Guided Finetuning Enhances Safety and Downstream Performance under Harmful Finetuning Attackscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Fine-Tuning Jailbreaks under Highly Constrained Black-Box Settings: A Three-Pronged Approach
  2. 2026Understanding and Preserving Safety in Fine-Tuned LLMs
  3. 2026SAFEGuard: Detect Optimization-Based Jailbreak Attacks Through Harmful Semantic Analysis and Fluency Measurement
  4. 2024Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Models
  5. 2024Jailbreak Antidote: Runtime Safety-Utility Balance via Sparse Representation Adjustment in Large Language Models
  6. 2025SafeInt: Shielding Large Language Models from Jailbreak Attacks via Safety-Aware Representation Intervention
  7. 2025Attack via Overfitting: 10-shot Benign Fine-tuning to Jailbreak LLMs
  8. 2026Defending Large Language Models Against Jailbreak Attacks via In-Decoding Safety-Awareness Probing