mlsanityData to 7 Oct 2026

Virus: Harmful Fine-tuning Attack for Large Language Models Bypassing Guardrail Moderation

arXiv 2501.17433 ↗First posted 29 Jan 2025cs.CR cs.AI cs.CLcited by 28

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Recent research shows that Large Language Models (LLMs) are vulnerable to harmful fine-tuning attacks -- models lose their safety alignment ability after fine-tuning on a few harmful samples. For risk mitigation, a guardrail is typically used to filter out harmful samples before fine-tuning. By designing a new red-teaming method, we in this paper show that purely relying on the moderation guardrail for data filtration is not reliable. Our proposed attack method, dubbed Virus, easily bypasses the guardrail moderation by slightly modifying the harmful data. Experimental results show that the harmful data optimized by Virus is not detectable by the guardrail with up to 100% leakage ratio, and can simultaneously achieve superior attack performance. Finally, the key message we want to convey through this paper is that: it is reckless to consider guardrail moderation as a clutch at straws towards harmful fine-tuning attack, as it cannot solve the inherent safety issue of the pre-trained LLMs. Our code is available at https://github.com/git-disl/Virus

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. First posted on arXiv 29 Jan 2025.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 18; 2026: 10.

28 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  2. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  3. 2025Benign Samples Matter! Fine-tuning On Outlier Benign Samples Severely Breaks Safetycited by 22
  4. 2025Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbationcited by 20
  5. 2025Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasetscited by 18
  6. 2025Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Schedulercited by 12
  7. 2025CTRAP: Embedding Collapse Trap to Safeguard Large Language Models from Harmful Fine-Tuningcited by 11
  8. 2025Shape it Up! Restoring LLM Safety during Finetuningcited by 11
  9. 2025Gradient Surgery for Safe LLM Fine-Tuningcited by 11
  10. 2025Unintended Misalignment from Agentic Fine-Tuning: Risks and Mitigationcited by 9
  11. 2025Safety Subspaces are Not Linearly Distinct: A Fine-Tuning Case Studycited by 6
  12. 2026Antibody: Strengthening Defense Against Harmful Fine-Tuning for Large Language Models via Attenuating Harmful Gradient Influencecited by 6
16 more
  1. 2025No, of Course I Can! Deeper Fine-Tuning Attacks That Bypass Token-Level Safety Mechanismscited by 5
  2. 2025Unified Defense for Large Language Models against Jailbreak and Fine-Tuning Attacks in Educationcited by 5
  3. 2025AntiDote: Bi-level Adversarial Training for Tamper-Resistant LLMscited by 4
  4. 2026Surgery: Mitigating Harmful Fine-Tuning for Large Language Models via Attention Sinkcited by 3
  5. 2026Token-level Data Selection for Safe LLM Fine-tuningcited by 3
  6. 2025Safety-Aligned Weights Are Not Enough: Refusal-Teacher-Guided Finetuning Enhances Safety and Downstream Performance under Harmful Finetuning Attackscited by 2
  7. 2025Circumventing Safety Alignment in Large Language Models Through Embedding Space Toxicity Attenuationcited by 2
  8. 2026Invisible Safety Threat: Malicious Finetuning for LLM via Steganographycited by 2
  9. 2025A Generative Approach to LLM Harmfulness Mitigation with Red Flag Tokenscited by 1
  10. 2026Lattice: Generative Guardrails for Conversational Agentscited by 1
  11. 2025Embedding Poisoning: Bypassing Safety Alignment via Embedding Semantic Shiftcited by 0
  12. 2026Learning to Stay Safe: Adaptive Regularization Against Safety Degradation during Fine-Tuningcited by 0
  13. 2026SelfGrader: LLM Jailbreak Detection via Anchored Token-Level Logitscited by 0
  14. 2026Defending Against Harmful Supervision Hidden in Benign Samplescited by 0
  15. 2026Yesterday's Shield, Today's Spear: A Self-Evolving Safety Guardrail in Productioncited by 0
  16. 2026Can Data Attribution Filter Out Subliminal Learning? Not Reliablycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Fine-Tuning Lowers Safety and Disrupts Evaluation Consistency
  2. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey
  3. 2026Security in the Fine-Tuning Lifecycle of Large Language Models: Threats, Defenses,Evaluation, and Future Directions
  4. 2024Overriding Safety protections of Open-source Models
  5. 2024Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning
  6. 2024No Two Devils Alike: Unveiling Distinct Mechanisms of Fine-tuning Attacks
  7. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!
  8. 2024Locking Down the Finetuned LLMs Safety