# Fine-Tuning Lowers Safety and Disrupts Evaluation Consistency

arXiv: https://arxiv.org/abs/2506.17209 · HTML: https://mlsanity.com/paper/2506.17209/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security.
- Builds on 15 of its 31 references that are in the map.
- First posted on arXiv 20 Jun 2025.

## Placed in

- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 15 of 15)

- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023)
- [Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models](https://mlsanity.com/paper/2310.02949/) (2023)
- [SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal](https://mlsanity.com/paper/2406.14598/) (2024)
- [Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory](https://mlsanity.com/paper/2310.17884/) (2023)
- [LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B](https://mlsanity.com/paper/2310.20624/) (2023)
- [Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs](https://mlsanity.com/paper/2502.17424/) (2025)
- [What is in Your Safe Data? Identifying Benign Data that Breaks Safety](https://mlsanity.com/paper/2404.01099/) (2024)
- [The Art of Saying No: Contextual Noncompliance in Language Models](https://mlsanity.com/paper/2407.12043/) (2024)
- [Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates](https://mlsanity.com/paper/2402.18540/) (2024)
- [MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notes](https://mlsanity.com/paper/2412.19260/) (2024)
- [SaLoRA: Safety-Alignment Preserved Low-Rank Adaptation](https://mlsanity.com/paper/2501.01765/) (2025)
- [Unveiling Safety Vulnerabilities of Large Language Models](https://mlsanity.com/paper/2311.04124/) (2023)
- [Goal-Oriented Prompt Attack and Safety Evaluation for LLMs](https://mlsanity.com/paper/2309.11830/) (2023)
- [Fine-Tuning, Quantization, and LLMs: Navigating Unintended Outcomes](https://mlsanity.com/paper/2404.04392/) (2024)
- [LLMs Lost in Translation: M-ALERT uncovers Cross-Linguistic Safety Inconsistencies](https://mlsanity.com/paper/2412.15035/) (2024)

## Built on by (most cited first, 27 of 27)

- [Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoring](https://mlsanity.com/paper/2512.12069/) (2025)
- [From Data to Behavior: Predicting Unintended Model Behaviors Before Training](https://mlsanity.com/paper/2602.04735/) (2026)
- [The Devil in the Details: Emergent Misalignment, Format and Coherence in Open-Weights LLMs](https://mlsanity.com/paper/2511.20104/) (2025)
- [The Great Pretender: A Stochasticity Problem in LLM Jailbreak](https://mlsanity.com/paper/2605.14418/) (2026)
- [MedRiskEval: Medical Risk Evaluation Benchmark of Language Models, On the Importance of User Perspectives in Healthcare Settings](https://mlsanity.com/paper/2507.07248/) (2025)
- [The Unintended Trade-off of AI Alignment:Balancing Hallucination Mitigation and Safety in LLMs](https://mlsanity.com/paper/2510.07775/) (2025)
- [Towards Identification and Intervention of Safety-Critical Parameters in Large Language Models](https://mlsanity.com/paper/2604.08297/) (2026)
- [The Problem with Safety Classification is not just the Models](https://mlsanity.com/paper/2507.21782/) (2025)
- [Steerability of Instrumental-Convergence Tendencies in LLMs](https://mlsanity.com/paper/2601.01584/) (2026)
- [The Hidden Costs of Domain Fine-Tuning: Pii-Bearing Data Degrades Safety and Increases Leakage](https://mlsanity.com/paper/2603.00061/) (2026)
- [Beyond Creed: A Non-Identity Safety Condition A Strong Empirical Alternative to Identity Framing in Low-Data LoRA Fine-Tuning](https://mlsanity.com/paper/2603.14723/) (2026)
- [RefusalGuard: Geometry-Preserving Fine-Tuning for Safety in LLMs](https://mlsanity.com/paper/2605.01913/) (2026)
- [SafeGene: Reusable Adapters for Transferable Safety Alignment](https://mlsanity.com/paper/2606.06519/) (2026)
- [Innocuous-Seeming Data, Latent Ideology: Ideological Generalisation in Finetuned LLMs](https://mlsanity.com/paper/2607.14888/) (2026)
- [Can Small-Scale Data Poisoning Exacerbate Dialect-Linked Biases in Large Language Models?](https://mlsanity.com/paper/2507.19195/) (2025)
- [Stemming Hallucination in Language Models Using a Licensing Oracle](https://mlsanity.com/paper/2511.06073/) (2025)
- [Modeling User-System Behavior for Training-free Building of Private Domain Conversational Agents](https://mlsanity.com/paper/2511.10283/) (2025)
- [Are Aligned Large Language Models Still Misaligned?](https://mlsanity.com/paper/2602.11305/) (2026)
- [GradShield: Alignment Preserving Finetuning](https://mlsanity.com/paper/2605.14194/) (2026)
- [Safety Measurements for Fine-tuned LLMs Should be Grounded in Capability](https://mlsanity.com/paper/2606.03648/) (2026)
- [The Heterogeneous Safety Impacts of Benign Multilingual Fine-Tuning](https://mlsanity.com/paper/2606.28843/) (2026)
- [HyperSafe: Inference-Time Safety Recovery for Fine-Tuned Language Models](https://mlsanity.com/paper/2607.11475/) (2026)
- [Trustworthiness Costs of Domain Adaptation in Small Language Models:A Cross-Architecture Empirical Study](https://mlsanity.com/paper/2608.00042/) (2026)
- [Safe Evolution with Circuit Anchors](https://mlsanity.com/paper/2608.05158/) (2026)
- [Off-Target Effects of Response-Style Alignment in a Korean 27B Language Model](https://mlsanity.com/paper/2609.11291/) (2026)
- [Alignment Inertia: Auditing the Durability of Training Data Influence Through Policy Override Resistance](https://mlsanity.com/paper/2609.27333/) (2026)
- [Mitigating Private Data Leakage in LLMs with Whiteout](https://mlsanity.com/paper/2610.02418/) (2026)

## Abstract (arXiv)

Fine-tuning a general-purpose large language model (LLM) for a specific domain or task has become a routine procedure for ordinary users. However, fine-tuning is known to remove the safety alignment features of the model, even when the fine-tuning data does not contain any harmful content. We consider this to be a critical failure mode of LLMs due to the widespread uptake of fine-tuning, combined with the benign nature of the "attack". Most well-intentioned developers are likely unaware that they are deploying an LLM with reduced safety. On the other hand, this known vulnerability can be easily exploited by malicious actors intending to bypass safety guardrails. To make any meaningful progress in mitigating this issue, we first need reliable and reproducible safety evaluations. In this work, we investigate how robust a safety benchmark is to trivial variations in the experimental procedure, and the stochastic nature of LLMs. Our initial experiments expose surprising variance in the results of the safety evaluation, even when seemingly inconsequential changes are made to the fine-tuning setup. Our observations have serious implications for how researchers in this field should report results to enable meaningful comparisons in the future.