mlsanityData to 7 Oct 2026

SDD: Self-Degraded Defense against Malicious Fine-tuning

arXiv 2507.21182 ↗First posted 27 Jul 2025cs.CR cs.AIcited by 13

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Open-source Large Language Models (LLMs) often employ safety alignment methods to resist harmful instructions. However, recent research shows that maliciously fine-tuning these LLMs on harmful data can easily bypass these safeguards. To counter this, we theoretically uncover why malicious fine-tuning succeeds and identify potential defense strategies. Building on the theoretical analysis, we introduce the Self-Degraded Defense (SDD) framework. SDD encourages LLMs to produce high-quality but irrelevant responses to harmful prompts. When attackers attempt malicious fine-tuning, the general capability of the LLM aligned by SDD will significantly decrease, rendering it incapable of following harmful instructions. Our experimental results confirm SDD's effectiveness against such attacks.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 41 of its 56 references that are in the map. First posted on arXiv 27 Jul 2025.

Builds on

41 of 56 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbation13 Oct
  2. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey26 Sep
  3. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deep10 Jun
  4. 2024Robustifying Safety-Aligned Large Language Models through Clean Data Curation24 May
  5. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications7 Feb
  6. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B31 Oct
  7. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedback19 Oct
  8. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
  9. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset10 Jul

Other references · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  10. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  11. 2020Learning to summarize from human feedbackcited by 2,562
  12. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
20 more
  1. 2019Invariant Risk Minimizationcited by 1,832
  2. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  3. 2023LIMA: Less Is More for Alignmentcited by 1,080
  4. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  5. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  6. 2020Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learningcited by 292
  7. 2020The Risks of Invariant Risk Minimizationcited by 262
  8. 2023Poisoning Language Models During Instruction Tuningcited by 254
  9. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 149
  10. 2024Tamper-Resistant Safeguards for Open-Weight LLMscited by 132
  11. 2023On the Exploitability of Instruction Tuningcited by 110
  12. 2023Some things are more CRINGE than others: Iterative Preference Optimization with the Pairwise Cringe Losscited by 108
  13. 2022Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLPcited by 99
  14. 2023BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicinecited by 99
  15. 2024Representation Noising: A Defence Mechanism Against Harmful Finetuningcited by 93
  16. 2024Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbationcited by 92
  17. 2022Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Modelscited by 56
  18. 2023BadLlama: cheaply removing safety fine-tuning from Llama 2-Chat 13Bcited by 35
  19. 2023Will releasing the weights of future large language models grant widespread access to pandemic agents?cited by 25
  20. 2023Language Model Unalignment: Parametric Red-Teaming to Expose Hidden Harms and Biasescited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 13.

13 papers.

Sort:

By area:

  1. 2026Surgery: Mitigating Harmful Fine-Tuning for Large Language Models via Attention Sinkcited by 3
  2. 2026Safeguarding LLM Fine-tuning via Push-Pull Distributional Alignmentcited by 2
  3. 2026GR-SAP: Generative Replay for Safety Alignment Preservation during Fine-Tuningcited by 2
  4. 2026TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tamperingcited by 1
  5. 2026Beyond Creed: A Non-Identity Safety Condition A Strong Empirical Alternative to Identity Framing in Low-Data LoRA Fine-Tuningcited by 1
  6. 2026AI Security in the Foundation Model Era: A Comprehensive Survey from a Unified Perspectivecited by 1
  7. 2026Immunizing 3D Gaussian Generative Models Against Unauthorized Fine-Tuning via Attribute-Space Trapscited by 1
  8. 2026A Few Steps Further: Why Defenses Against Malicious Finetuning Erode Under Continued Trainingcited by 1
  9. 2026Open-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attackscited by 1
  10. 2026Alignment-Aware Model Adaptation via Feedback-Guided Optimizationcited by 0
  11. 2026GradShield: Alignment Preserving Finetuningcited by 0
  12. 2026Detect, Unlearn, Restore: Defending Text Summarization Models Against Data Poisoningcited by 0
1 more
  1. 2026Preemptive LLM Unlearning against Forbidden Capability Acquisition via Gradient Sealingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Security in the Fine-Tuning Lifecycle of Large Language Models: Threats, Defenses,Evaluation, and Future Directions
  2. 2025Fine-Tuning Lowers Safety and Disrupts Evaluation Consistency
  3. 2024No Two Devils Alike: Unveiling Distinct Mechanisms of Fine-tuning Attacks
  4. 2025CTRAP: Embedding Collapse Trap to Safeguard Large Language Models from Harmful Fine-Tuning
  5. 2025Decoding One Safety Trigger Token for Balancing Safety and Usability in Large Language Models
  6. 2024Root Defence Strategies: Ensuring Safety of LLM at the Decoding Level
  7. 2026Understanding and Preserving Safety in Fine-Tuned LLMs
  8. 2024Safety Layers in Aligned Large Language Models: The Key to LLM Security