Data to 7 Oct 2026

Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbation

arXiv 2409.01586 ↗First posted 3 Sep 2024cs.CL cs.AIcited by 92

Not on the map. Why?

Cited by 57 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Harmful fine-tuning attack poses serious safety concerns for large language models' fine-tuning-as-a-service. While existing defenses have been proposed to mitigate the issue, their performances are still far away from satisfactory, and the root cause of the problem has not been fully recovered. To this end, we in this paper show that harmful perturbation over the model weights could be a probable cause of alignment-broken. In order to attenuate the negative impact of harmful perturbation, we propose an alignment-stage solution, dubbed Booster. Technically, along with the original alignment loss, we append a loss regularizer in the alignment stage's optimization. The regularizer ensures that the model's harmful loss reduction after the simulated harmful perturbation is attenuated, thereby mitigating the subsequent fine-tuning risk. Empirical results show that Booster can effectively reduce the harmful score of the fine-tuned models while maintaining the performance of downstream tasks. Our code is available at https://github.com/git-disl/Booster.

Key facts. Cited by 92 papers in the map (data through 7 Oct 2026). Builds on 56 of its 112 references that are in the map. First posted on arXiv 3 Sep 2024.

Builds on

56 of 112 references.

References · 56

2022Training language models to follow instructions with human feedbackcited by 15,712
2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
2021Training Verifiers to Solve Math Word Problemscited by 9,743
2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
2015Character-level Convolutional Networks for Text Classificationcited by 2,293
2024Qwen2 Technical Reportcited by 2,076
2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
2017Decoupled Weight Decay Regularizationcited by 1,722
2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106

38 more

2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
2019Meta-Learning with Implicit Gradientscited by 523
2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 454
2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
2024Improving Alignment and Robustness with Circuit Breakerscited by 302
2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
2023LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Modelscited by 195
2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 183
2023Removing RLHF Protections in GPT-4 via Fine-Tuningcited by 165
2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 160
2024Tamper-Resistant Safeguards for Open-Weight LLMscited by 132
2024Safety Layers in Aligned Large Language Models: The Key to LLM Securitycited by 126
2022Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learningcited by 122
2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 122
2024What is in Your Safe Data? Identifying Benign Data that Breaks Safetycited by 114
2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 113
2024BadAgent: Inserting and Activating Backdoor Attacks in LLM Agentscited by 106
2024Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmeticcited by 105
2024Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templatescited by 94
2023Fine-tuning can cripple your foundation model; preserving features may be the solutioncited by 92
2024Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 89
2024Defending Against Unforeseen Failure Modes with Latent Adversarial Trainingcited by 82
2024Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptationcited by 74
2024Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuningcited by 71
2022Boosting the Transferability of Adversarial Attacks with Reverse Adversarial Perturbationcited by 67
2023Pairwise Proximal Policy Optimization: Harnessing Relative Feedback for LLM Alignmentcited by 57
2023Large Language Model-Powered Smart Contract Vulnerability Detection: New Perspectivescited by 48
2024ALI-Agent: Assessing LLMs' Alignment with Human Values via Agent-based Evaluationcited by 46
2024Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Modelscited by 45
2024Open Problems in Technical AI Governancecited by 45
2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
2024Robustifying Safety-Aligned Large Language Models through Clean Data Curationcited by 43
2024A safety realignment framework via subspace-oriented model fusion for large language modelscited by 40

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 44; 2026: 41.

92 papers.

Sort:

By area:

2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 86
2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
2024On Evaluating the Durability of Safeguards for Open-Weight LLMscited by 51
2024Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbationcited by 35
2025SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Mergingcited by 25
2025Benign Samples Matter! Fine-tuning On Outlier Benign Samples Severely Breaks Safetycited by 22
2025Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbationcited by 20
2025Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasetscited by 18
2025AsFT: Anchoring Safety During LLM Fine-Tuning Within Narrow Safety Basincited by 18
2025Safe Delta: Consistently Preserving Safety when Fine-Tuning LLMs on Diverse Datasetscited by 17
2025Vulnerability-Aware Alignment: Mitigating Uneven Forgetting in Harmful Fine-Tuningcited by 17
2025Estimating Worst-Case Frontier Risks of Open-Weight LLMscited by 17

38 more

2025Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Modelscited by 15
2024Superficial Safety Alignment Hypothesiscited by 14
2025SDD: Self-Degraded Defense against Malicious Fine-tuningcited by 13
2025LoX: Low-Rank Extrapolation Robustifies LLM Safety Against Fine-tuningcited by 12
2025Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Schedulercited by 12
2024H3Fusion: Helpful, Harmless, Honest Fusion of Aligned LLMscited by 11
2025Scalable Fingerprinting of Large Language Modelscited by 11
2025CTRAP: Embedding Collapse Trap to Safeguard Large Language Models from Harmful Fine-Tuningcited by 11
2025Shape it Up! Restoring LLM Safety during Finetuningcited by 11
2025Gradient Surgery for Safe LLM Fine-Tuningcited by 11
2025Self-Destructive Language Modelcited by 10
2025Unforgotten Safety: Preserving Safety Alignment of Large Language Models with Continual Learningcited by 10
2025Too Helpful, Too Harmless, Too Honest or Just Right?cited by 9
2025A Guardrail for Safety Preservation: When Safety-Sensitive Subspace Meets Harmful-Resistant Null-Spacecited by 9
2024Toward Secure Tuning: Mitigating Security Risks from Instruction Fine-Tuningcited by 8
2025Towards Resilient Safety-driven Unlearning for Diffusion Models against Downstream Fine-tuningcited by 8
2025Defending MoE LLMs against Harmful Fine-Tuning via Safety Routing Alignmentcited by 8
2025MetaDefense: Defending Finetuning-based Jailbreak Attack Before and During Generationcited by 8
2025Toward Robust Non-Transferable Learning: A Survey and Benchmarkcited by 7
2025Token Buncher: Shielding LLMs from Harmful Reinforcement Learning Fine-Tuningcited by 7
2025Safety Subspaces are Not Linearly Distinct: A Fine-Tuning Case Studycited by 6
2025When Style Breaks Safety: Defending LLMs Against Superficial Style Alignmentcited by 6
2025Anchoring Refusal Direction: Mitigating Safety Risks in Tuning via Projection Constraintcited by 6
2025Pharmacist: Safety Alignment Data Curation for Large Language Models against Harmful Fine-tuningcited by 6
2026Antibody: Strengthening Defense Against Harmful Fine-Tuning for Large Language Models via Attenuating Harmful Gradient Influencecited by 6
2024Recent Advances in Attack and Defense Approaches of Large Language Modelscited by 5
2025No, of Course I Can! Deeper Fine-Tuning Attacks That Bypass Token-Level Safety Mechanismscited by 5
2025Erase or Hide? Suppressing Spurious Unlearning Neurons for Robust Unlearningcited by 5
2025Unified Defense for Large Language Models against Jailbreak and Fine-Tuning Attacks in Educationcited by 5
2025Offline Learning and Forgetting for Reasoning with Large Language Modelscited by 4
2025Watch your steps: Dormant Adversarial Behaviors that Activate upon LLM Finetuningcited by 4
2025SC-LoRA: Balancing Efficient Fine-tuning and Knowledge Preservation via Subspace-Constrained LoRAcited by 4
2026Continual Safety Alignment via Gradient-Based Sample Selectioncited by 4
2025Fight Fire with Fire: Defending Against Malicious RL Fine-Tuning via Reward Neutralizationcited by 3
2026Large Language Models Generate Harmful Responses Using a Distinct Mechanism, Shared Across Harm Typescited by 3
2025Safety-Aligned Weights Are Not Enough: Refusal-Teacher-Guided Finetuning Enhances Safety and Downstream Performance under Harmful Finetuning Attackscited by 2
2025The Unintended Trade-off of AI Alignment:Balancing Hallucination Mitigation and Safety in LLMscited by 2
2025Learning from the Undesirable: Robust Adaptation of Language Models without Forgettingcited by 2

50 most cited of 92 All 92 (JSON)