mlsanityData to 14 Oct 2026

GradShield: Alignment Preserving Finetuning

arXiv 2605.14194 ↗First posted 13 May 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) pose a significant risk of safety misalignment after finetuning, as models can be compromised by both explicitly and implicitly harmful data. Even some seemingly benign data can inadvertently steer a model towards misaligned behaviors. To address this, we introduce GradShield, a principled filtering method that safeguards LLMs during finetuning by identifying and removing harmful data points before they corrupt the model's alignment. It removes potentially harmful data by computing a Finetuning Implicit Harmfulness Score (FIHS) for each data point and employs an adaptive thresholding algorithm. We apply GradShield to multiple utility fine-tuning tasks across varying levels of harmful data and evaluate the safety and utility performance of the resulting LLMs using various metrics. The results show that GradShield outperforms all baseline methods, consistently maintaining an Attack Success Rate (ASR) below 6% while preserving utility performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 42 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

38 of 42 references.

References · 38

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  9. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  10. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  11. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  12. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 580
26 more
  1. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 535
  2. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 488
  3. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 472
  4. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
  5. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 173
  6. 2023Removing RLHF Protections in GPT-4 via Fine-Tuningcited by 171
  7. 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 125
  8. 2024What is in Your Safe Data? Identifying Benign Data that Breaks Safetycited by 118
  9. 2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 115
  10. 2023Fine-tuning can cripple your foundation model; preserving features may be the solutioncited by 96
  11. 2024Representation Noising: A Defence Mechanism Against Harmful Finetuningcited by 96
  12. 2024Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 92
  13. 2024Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbationcited by 92
  14. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 90
  15. 2024Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptationcited by 80
  16. 2024Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuningcited by 72
  17. 2024SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selectioncited by 64
  18. 2024Robustifying Safety-Aligned Large Language Models through Clean Data Curationcited by 44
  19. 2025SaLoRA: Safety-Alignment Preserved Low-Rank Adaptationcited by 42
  20. 2024NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuningcited by 37
  21. 2024Safety-Aware Fine-Tuning of Large Language Modelscited by 34
  22. 2025Fine-Tuning Lowers Safety and Disrupts Evaluation Consistencycited by 27
  23. 2025Benign Samples Matter! Fine-tuning On Outlier Benign Samples Severely Breaks Safetycited by 22
  24. 2024Do Influence Functions Work on Large Language Models?cited by 20
  25. 2024Toxicity Detection for Freecited by 15
  26. 2025SDD: Self-Degraded Defense against Malicious Fine-tuningcited by 13

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Gradient Surgery for Safe LLM Fine-Tuning
  2. 2024Separate the Wheat from the Chaff: A Post-Hoc Approach to Safety Re-Alignment for Fine-Tuned Language Models
  3. 2024Towards Comprehensive Post Safety Alignment of Large Language Models via Safety Patching
  4. 2025SafeTuneBed: A Toolkit for Benchmarking LLM Safety Alignment in Fine-Tuning
  5. 2026Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instance
  6. 2026The Art of (Mis)alignment: How Fine-Tuning Methods Effectively Misalign and Realign LLMs in Post-Training
  7. 2024Locking Down the Finetuned LLMs Safety
  8. 2025SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging