mlsanityData to 7 Oct 2026

Toward Secure Tuning: Mitigating Security Risks from Instruction Fine-Tuning

arXiv 2410.04524 ↗First posted 6 Oct 2024cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Instruction fine-tuning has emerged as a critical technique for customizing Large Language Models (LLMs) to specific applications. However, recent studies have highlighted significant security vulnerabilities in fine-tuned LLMs. Existing defense efforts focus more on pre-training and post-training methods, yet there remains underexplored in in-training methods. To fill this gap, we introduce a novel secure-tuning strategy called SWAT. By analyzing how module-level parameters (e.g. Q/K/V/O) affect the security feature space drift, we identify a robust subset of modules, termed Mods_Rob. Our SWAT strategy begins by warming up Mods_Rob to capture low-level features with minimal security risks, followed by training all parameters to achieve optimal task performance. Essentially, this strategy shifts the early learning burden more from global parameters to Mods_Rob, reducing update magnitudes of the non-robust subset. Across various datasets, scenarios, and LLMs, our strategy has demonstrated significant success in mitigating security risks while preserving task performance. Importantly, it can be seamlessly integrated with pre-training and post-training methods, leading to greater improvements.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 32 of its 45 references that are in the map. First posted on arXiv 6 Oct 2024.

Builds on

32 of 45 references.

References · 32

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  8. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  10. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  11. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  12. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 679
20 more
  1. 2019Importance Estimation for Neural Network Pruningcited by 531
  2. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  3. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  4. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
  5. 2024Advancing LLM Reasoning Generalists with Preference Treescited by 191
  6. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 162
  7. 2024Safety Layers in Aligned Large Language Models: The Key to LLM Securitycited by 126
  8. 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 122
  9. 2023Explore, Establish, Exploit: Red Teaming Language Models from Scratchcited by 111
  10. 2024Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmeticcited by 105
  11. 2023Fine-tuning can cripple your foundation model; preserving features may be the solutioncited by 92
  12. 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden Statescited by 92
  13. 2024Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbationcited by 92
  14. 2024Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 89
  15. 2024AgentInstruct: Toward Generative Teaching with Agentic Flowscited by 88
  16. 2024Robustifying Safety-Aligned Large Language Models through Clean Data Curationcited by 43
  17. 2024Instruction Tuning With Loss Over Instructionscited by 40
  18. 2024SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuningcited by 32
  19. 2024Foot In The Door: Understanding Large Language Model Jailbreaking via Cognitive Psychologycited by 25
  20. 2024MoGU: A Framework for Enhancing Safety of Open-Sourced LLMs While Preserving Their Usabilitycited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 4.

8 papers.

  1. 2024Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbationcited by 35
  2. 2025Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasetscited by 18
  3. 2025Self-Destructive Language Modelcited by 10
  4. 2026Antibody: Strengthening Defense Against Harmful Fine-Tuning for Large Language Models via Attenuating Harmful Gradient Influencecited by 6
  5. 2026Surgery: Mitigating Harmful Fine-Tuning for Large Language Models via Attention Sinkcited by 3
  6. 2026Alignment-Weighted DPO: A principled reasoning approach to improve safety alignmentcited by 2
  7. 2025SafeTuneBed: A Toolkit for Benchmarking LLM Safety Alignment in Fine-Tuningcited by 1
  8. 2026TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tamperingcited by 1