# Here's a Free Lunch: Sanitizing Backdoored Models with Model Merge

arXiv: https://arxiv.org/abs/2402.19334 · HTML: https://mlsanity.com/paper/2402.19334/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 50 references that are in the map.
- First posted on arXiv 29 Feb 2024.

## Builds on (most cited first, 30 of 34)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time](https://mlsanity.com/paper/2203.05482/) (2022)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [The LAMBADA dataset: Word prediction requiring a broad discourse context](https://mlsanity.com/paper/1606.06031/) (2016)
- [Merging Models with Fisher-Weighted Averaging](https://mlsanity.com/paper/2111.09832/) (2021)
- [Dataless Knowledge Fusion by Merging Weights of Language Models](https://mlsanity.com/paper/2212.09849/) (2022)
- [Weight Poisoning Attacks on Pre-trained Models](https://mlsanity.com/paper/2004.06660/) (2020)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [Poisoning Language Models During Instruction Tuning](https://mlsanity.com/paper/2305.00944/) (2023)
- [ONION: A Simple and Effective Defense Against Textual Backdoor Attacks](https://mlsanity.com/paper/2011.10369/) (2020)
- [Anti-Backdoor Learning: Training Clean Models on Poisoned Data](https://mlsanity.com/paper/2110.11571/) (2021)
- [A backdoor attack against LSTM-based text classification systems](https://mlsanity.com/paper/1905.12457/) (2019)
- [Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger](https://mlsanity.com/paper/2105.12400/) (2021)
- [BackdoorBench: A Comprehensive Benchmark of Backdoor Learning](https://mlsanity.com/paper/2206.12654/) (2022)
- [Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoning](https://mlsanity.com/paper/2108.13888/) (2021)
- [Fusing finetuned models for better pretraining](https://mlsanity.com/paper/2204.03044/) (2022)
- [Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution](https://mlsanity.com/paper/2106.06361/) (2021)
- [BITE: Textual Backdoor Attacks with Iterative Trigger Injection](https://mlsanity.com/paper/2205.12700/) (2022)
- [Analyzing And Editing Inner Mechanisms Of Backdoored Language Models](https://mlsanity.com/paper/2302.12461/) (2023)

## Built on by (most cited first, 26 of 26)

- [Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)](https://mlsanity.com/paper/2407.14937/) (2024)
- [BadMerging: Backdoor Attacks Against Model Merging](https://mlsanity.com/paper/2408.07362/) (2024)
- [Unlearning Backdoor Attacks for LLMs with Weak-to-Strong Knowledge Distillation](https://mlsanity.com/paper/2410.14425/) (2024)
- [Copyright Protection for Large Language Models: A Survey of Methods, Challenges, and Trends](https://mlsanity.com/paper/2508.11548/) (2025)
- [Have You Merged My Model? On The Robustness of Large Language Model IP Protection Methods Against Model Merging](https://mlsanity.com/paper/2404.05188/) (2024)
- [BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judge](https://mlsanity.com/paper/2503.00596/) (2025)
- [Mitigating Backdoor Threats to Large Language Models: Advancement and Challenges](https://mlsanity.com/paper/2409.19993/) (2024)
- [Mitigating the Backdoor Effect for Multi-Task Model Merging via Safety-Aware Subspace](https://mlsanity.com/paper/2410.13910/) (2024)
- [Rethinking Backdoor Detection Evaluation for Language Models](https://mlsanity.com/paper/2409.00399/) (2024)
- [Gracefully Filtering Backdoor Samples for Generative Large Language Models without Retraining](https://mlsanity.com/paper/2412.02454/) (2024)
- [Merge Hijacking: Backdoor Attacks to Model Merging of Large Language Models](https://mlsanity.com/paper/2505.23561/) (2025)
- [Be Careful When Fine-tuning On Open-Source LLMs: Your Fine-tuning Data Could Be Secretly Stolen!](https://mlsanity.com/paper/2505.15656/) (2025)
- [Patronus: Identifying and Mitigating Transferable Backdoors in Pre-trained Language Models](https://mlsanity.com/paper/2512.06899/) (2025)
- [BADTV: Unveiling Backdoor Threats in Third-Party Task Vectors](https://mlsanity.com/paper/2501.02373/) (2025)
- [Merge Now, Regret Later: The Hidden Cost of Model Merging Is Adversarial Transferability](https://mlsanity.com/paper/2509.23689/) (2025)
- [Backdoor Vectors: a Task Arithmetic View on Backdoor Attacks and Defenses](https://mlsanity.com/paper/2510.08016/) (2025)
- [Do Not Merge My Model! Safeguarding Open-Source LLMs Against Unauthorized Model Merging](https://mlsanity.com/paper/2511.10712/) (2025)
- [Cut the Deadwood Out: Backdoor Purification via Guided Module Substitution](https://mlsanity.com/paper/2412.20476/) (2024)
- [Lethe: Purifying Backdoored Large Language Models with Knowledge Dilution](https://mlsanity.com/paper/2508.21004/) (2025)
- [GREAT: Generalizable Backdoor Attacks in RLHF via Emotion-Aware Trigger Synthesis](https://mlsanity.com/paper/2510.09260/) (2025)
- [Towards Poisoning Robustness Certification for Natural Language Generation](https://mlsanity.com/paper/2602.09757/) (2026)
- [BackFlush: Knowledge-Free Backdoor Detection and Elimination with Watermark Preservation in Large Language Models](https://mlsanity.com/paper/2605.12529/) (2026)
- [Protecting Copyright of Medical Pre-trained Language Models: Training-Free Backdoor Model Watermarking](https://mlsanity.com/paper/2409.10570/) (2024)
- [Defending against Backdoor Attacks via Module Switching](https://mlsanity.com/paper/2504.05902/) (2025)
- [DiffSafeMerge: Mitigating Backdoor Inheritance in Diffusion Model Merging](https://mlsanity.com/paper/2608.09445/) (2026)
- [Towards Merging-conditional Backdoor Unalignment against Large Language Models](https://mlsanity.com/paper/2610.14701/) (2026)

## Abstract (arXiv)

The democratization of pre-trained language models through open-source initiatives has rapidly advanced innovation and expanded access to cutting-edge technologies. However, this openness also brings significant security risks, including backdoor attacks, where hidden malicious behaviors are triggered by specific inputs, compromising natural language processing (NLP) system integrity and reliability. This paper suggests that merging a backdoored model with other homogeneous models can significantly remediate backdoor vulnerabilities even if such models are not entirely secure. In our experiments, we verify our hypothesis on various models (BERT-Base, RoBERTa-Large, Llama2-7B, and Mistral-7B) and datasets (SST-2, OLID, AG News, and QNLI). Compared to multiple advanced defensive approaches, our method offers an effective and efficient inference-stage defense against backdoor attacks on classification and instruction-tuned tasks without additional resources or specific knowledge. Our approach consistently outperforms recent advanced baselines, leading to an average of about 75% reduction in the attack success rate. Since model merging has been an established approach for improving model performance, the extra advantage it provides regarding defense can be seen as a cost-free bonus.