mlsanityData to 7 Oct 2026

SPARD: Defending Harmful Fine-Tuning Attack via Safety Projection with Relevance-Diversity Data Selection

arXiv 2605.28030 ↗First posted 27 May 2026cs.LG cs.AI cs.CRcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Fine-tuning large language models often undermines their safety alignment, a problem further amplified by harmful fine-tuning attacks in which adversarial data removes safeguards and induces unsafe behaviors. We propose SPARD, a defense framework that integrates Safety-Projected Alternating optimization with Relevance-Diversity aware data selection. SPARD employs SPAG, which optimizes alternatively between utility updates and explicit safety projections with a set of safe data to enforce safety constraints. To curate safe data, we introduce a Relevance-Diversity Determinantal Point Process to select compact safe data, balancing task relevance and safety coverage. Experiments on GSM8K and OpenBookQA under four harmful fine-tuning attacks demonstrate that SPARD consistently achieves the lowest average attack success rates, substantially outperforming state-of-the-art defense methods, while maintaining high task accuracy. Code is available at https://github.com/shuhao02/SPARD.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 49 of its 62 references that are in the map. First posted on arXiv 27 May 2026.

Builds on

49 of 62 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025Pharmacist: Safety Alignment Data Curation for Large Language Models against Harmful Fine-tuning11 Oct
  2. 2025Gradient Surgery for Safe LLM Fine-Tuning10 Aug
  3. 2025Vulnerability-Aware Alignment: Mitigating Uneven Forgetting in Harmful Fine-Tuning4 Jun
  4. 2024Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbation13 Oct
  5. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey26 Sep
  6. 2024Do as I do (Safely): Mitigating Task-Specific Fine-tuning Risks in Large Language Models12 Jun
  7. 2024Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attack28 May
  8. 2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attack2 Feb
  9. 2023Removing RLHF Protections in GPT-4 via Fine-Tuning9 Nov
  10. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B31 Oct
  11. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
  12. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models4 Oct
3 more
  1. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  3. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset10 Jul

Other references · 34

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2024Qwen2.5 Technical Reportcited by 4,713
  7. 2024GPT-4o System Cardcited by 4,467
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  9. 2015Trust Region Policy Optimizationcited by 2,503
  10. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  11. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  12. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
22 more
  1. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  2. 2023DataComp: In search of the next generation of multimodal datasetscited by 635
  3. 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 519
  4. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 514
  5. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  6. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 454
  7. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  8. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  9. 2024A Survey on Data Selection for Language Modelscited by 230
  10. 2023A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicitycited by 180
  11. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  12. 2023Compositional Exemplars for In-context Learningcited by 143
  13. 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 122
  14. 2024Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templatescited by 94
  15. 2024Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbationcited by 92
  16. 2024RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Modelscited by 84
  17. 2024Repetition Improves Language Model Embeddingscited by 70
  18. 2023Forward-Backward Reasoning in Large Language Models for Mathematical Verificationcited by 39
  19. 2024GITA: Graph to Visual and Textual Integration for Vision-Language Graph Reasoningcited by 31
  20. 2025PARM: Multi-Objective Test-Time Alignment via Preference-Aware Autoregressive Reward Modelcited by 27
  21. 2023Effective Structured Prompting by Meta-Learning and Representative Verbalizercited by 14
  22. 2025MetaDefense: Defending Finetuning-based Jailbreak Attack Before and During Generationcited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Safety Anchor: Defending Harmful Fine-tuning via Geometric Bottlenecks
  2. 2025Secure LLM Fine-Tuning via Safety-Aware Probing
  3. 2026Understanding and Preserving Safety in Fine-Tuned LLMs
  4. 2025Rethinking Safety in LLM Fine-tuning: An Optimization Perspective
  5. 2025SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging
  6. 2026Antibody: Strengthening Defense Against Harmful Fine-Tuning for Large Language Models via Attenuating Harmful Gradient Influence
  7. 2026Gradient Immunity: Null-Space Resistance to Malicious Fine-Tuning
  8. 2025Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation