mlsanityData to 14 Oct 2026

Fine-Tuning Lowers Safety and Disrupts Evaluation Consistency

arXiv 2506.17209 ↗First posted 20 Jun 2025cs.CLcited by 27

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Fine-tuning a general-purpose large language model (LLM) for a specific domain or task has become a routine procedure for ordinary users. However, fine-tuning is known to remove the safety alignment features of the model, even when the fine-tuning data does not contain any harmful content. We consider this to be a critical failure mode of LLMs due to the widespread uptake of fine-tuning, combined with the benign nature of the "attack". Most well-intentioned developers are likely unaware that they are deploying an LLM with reduced safety. On the other hand, this known vulnerability can be easily exploited by malicious actors intending to bypass safety guardrails. To make any meaningful progress in mitigating this issue, we first need reliable and reproducible safety evaluations. In this work, we investigate how robust a safety benchmark is to trivial variations in the experimental procedure, and the stochastic nature of LLMs. Our initial experiments expose surprising variance in the results of the safety evaluation, even when seemingly inconsequential changes are made to the fine-tuning setup. Our observations have serious implications for how researchers in this field should report results to enable meaningful comparisons in the future.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 15 of its 31 references that are in the map. First posted on arXiv 20 Jun 2025.

Builds on

15 of 31 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs24 Feb
  2. 2025SaLoRA: Safety-Alignment Preserved Low-Rank Adaptation3 Jan
  3. 2024Fine-Tuning, Quantization, and LLMs: Navigating Unintended Outcomes5 Apr
  4. 2024What is in Your Safe Data? Identifying Benign Data that Breaks Safety1 Apr
  5. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B31 Oct
  6. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct

Other references · 9

  1. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 297
  2. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 240
  3. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 230
  4. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 105
  5. 2024Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templatescited by 96
  6. 2024MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notescited by 80
  7. 2023Unveiling Safety Vulnerabilities of Large Language Modelscited by 27
  8. 2023Goal-Oriented Prompt Attack and Safety Evaluation for LLMscited by 24
  9. 2024LLMs Lost in Translation: M-ALERT uncovers Cross-Linguistic Safety Inconsistenciescited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 19.

27 papers.

Sort:

By area:

  1. 2025Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoringcited by 11
  2. 2026From Data to Behavior: Predicting Unintended Model Behaviors Before Trainingcited by 8
  3. 2025The Devil in the Details: Emergent Misalignment, Format and Coherence in Open-Weights LLMscited by 5
  4. 2026The Great Pretender: A Stochasticity Problem in LLM Jailbreakcited by 3
  5. 2025MedRiskEval: Medical Risk Evaluation Benchmark of Language Models, On the Importance of User Perspectives in Healthcare Settingscited by 2
  6. 2025The Unintended Trade-off of AI Alignment:Balancing Hallucination Mitigation and Safety in LLMscited by 2
  7. 2026Towards Identification and Intervention of Safety-Critical Parameters in Large Language Modelscited by 2
  8. 2025The Problem with Safety Classification is not just the Modelscited by 1
  9. 2026Steerability of Instrumental-Convergence Tendencies in LLMscited by 1
  10. 2026The Hidden Costs of Domain Fine-Tuning: Pii-Bearing Data Degrades Safety and Increases Leakagecited by 1
  11. 2026Beyond Creed: A Non-Identity Safety Condition A Strong Empirical Alternative to Identity Framing in Low-Data LoRA Fine-Tuningcited by 1
  12. 2026RefusalGuard: Geometry-Preserving Fine-Tuning for Safety in LLMscited by 1
15 more
  1. 2026SafeGene: Reusable Adapters for Transferable Safety Alignmentcited by 1
  2. 2026Innocuous-Seeming Data, Latent Ideology: Ideological Generalisation in Finetuned LLMscited by 1
  3. 2025Can Small-Scale Data Poisoning Exacerbate Dialect-Linked Biases in Large Language Models?cited by 0
  4. 2025Stemming Hallucination in Language Models Using a Licensing Oraclecited by 0
  5. 2025Modeling User-System Behavior for Training-free Building of Private Domain Conversational Agentscited by 0
  6. 2026Are Aligned Large Language Models Still Misaligned?cited by 0
  7. 2026GradShield: Alignment Preserving Finetuningcited by 0
  8. 2026Safety Measurements for Fine-tuned LLMs Should be Grounded in Capabilitycited by 0
  9. 2026The Heterogeneous Safety Impacts of Benign Multilingual Fine-Tuningcited by 0
  10. 2026HyperSafe: Inference-Time Safety Recovery for Fine-Tuned Language Modelscited by 0
  11. 2026Trustworthiness Costs of Domain Adaptation in Small Language Models:A Cross-Architecture Empirical Studycited by 0
  12. 2026Safe Evolution with Circuit Anchorscited by 0
  13. 2026Off-Target Effects of Response-Style Alignment in a Korean 27B Language Modelcited by 0
  14. 2026Alignment Inertia: Auditing the Durability of Training Data Influence Through Policy Override Resistancecited by 0
  15. 2026Mitigating Private Data Leakage in LLMs with Whiteoutcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Understanding and Preserving Safety in Fine-Tuned LLMs
  2. 2025Rethinking Safety in LLM Fine-tuning: An Optimization Perspective
  3. 2025Secure LLM Fine-Tuning via Safety-Aware Probing
  4. 2024What Makes and Breaks Safety Fine-tuning? A Mechanistic Study
  5. 2026From Parameter Dynamics to Risk Scoring : Quantifying Sample-Level Safety Degradation in LLM Fine-tuning
  6. 2024Locking Down the Finetuned LLMs Safety
  7. 2025LLM-Safety Evaluations Lack Robustness
  8. 2024Multitask Mayhem: Unveiling and Mitigating Safety Gaps in LLMs Fine-tuning