mlsanityData to 14 Oct 2026

Safeguard Fine-Tuned LLMs Through Pre- and Post-Tuning Model Merging

arXiv 2412.19512 ↗First posted 27 Dec 2024cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

Fine-tuning large language models (LLMs) for downstream tasks often leads to catastrophic forgetting, notably degrading the safety of originally aligned models. While some existing methods attempt to restore safety by incorporating additional safety data, the quality of such data typically falls short of that used in the original alignment process. Moreover, these high-quality safety datasets are generally inaccessible, making it difficult to fully recover the model's original safety. We ask: How can we preserve safety while improving downstream task performance without additional safety data? We show that simply merging the weights of pre- and post-fine-tuned models effectively mitigates safety degradation while enhancing performance. Experiments across different downstream tasks and models validate the method's practicality and effectiveness.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 31 of its 45 references that are in the map. First posted on arXiv 27 Dec 2024.

Builds on

31 of 45 references.

References · 31

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  6. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  7. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  8. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  9. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  10. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  11. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  12. 2021Robust fine-tuning of zero-shot modelscited by 797
19 more
  1. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  2. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  3. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 694
  4. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  5. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 496
  6. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
  7. 2023ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledgecited by 325
  8. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 297
  9. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 256
  10. 2024Arcee's MergeKit: A Toolkit for Merging Large Language Modelscited by 194
  11. 2024Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmeticcited by 112
  12. 2022Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLPcited by 102
  13. 2024Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuningcited by 96
  14. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 95
  15. 2024Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 92
  16. 2024Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuningcited by 72
  17. 2023Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languagescited by 54
  18. 2024Safety Arithmetic: A Framework for Test-time Safety Alignment of Language Models by Steering Parameters and Activationscited by 32
  19. 2024Separate the Wheat from the Chaff: A Post-Hoc Approach to Safety Re-Alignment for Fine-Tuned Language Modelscited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 7.

11 papers.

  1. 2025SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Mergingcited by 25
  2. 2025Layer-Aware Representation Filtering: Purifying Finetuning Data to Preserve LLM Safety Alignmentcited by 24
  3. 2026Safety Alignment as Continual Learning: Mitigating the Alignment Tax via Orthogonal Gradient Projectioncited by 6
  4. 2025Merge Now, Regret Later: The Hidden Cost of Model Merging Is Adversarial Transferabilitycited by 2
  5. 2026GR-SAP: Generative Replay for Safety Alignment Preservation during Fine-Tuningcited by 2
  6. 2026SafeGene: Reusable Adapters for Transferable Safety Alignmentcited by 1
  7. 2025The Forgotten Shield: Safety Grafting in Parameter-Space for Medical MLLMscited by 0
  8. 2026You Snooze, You Lose: Automatic Safety Alignment Restoration through Neural Weight Translationcited by 0
  9. 2026Leveraging Instruction Tuning and Merging for Reasoning Model Adaptationcited by 0
  10. 2026When Skills Meet Safety: Benchmarking and Characterizing the Adaptive Jailbreak Robustness of Skill-Merged LLMscited by 0
  11. 2026SafeTune: A Unified Faithful Library for Auditing and Repairing Safety Drift in Fine-Tuned LLMscited by 0