# CleanGen: Mitigating Backdoor Attacks for Generation Tasks in Large Language Models

arXiv: https://arxiv.org/abs/2406.12257 · HTML: https://mlsanity.com/paper/2406.12257/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security.
- Builds on 29 of its 44 references that are in the map.
- First posted on arXiv 18 Jun 2024.

## Placed in

- LLM safety and security / [Backdoor and data poisoning attacks](https://mlsanity.com/safety/backdoor-and-data-poisoning-attacks/)

## Builds on (most cited first, 29 of 29)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023)
- [Enhancing Chat Language Models by Scaling High-quality Instructional Conversations](https://mlsanity.com/paper/2305.14233/) (2023)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks](https://mlsanity.com/paper/1805.12185/) (2018)
- [Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training](https://mlsanity.com/paper/2401.05566/) (2024)
- [SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding](https://mlsanity.com/paper/2402.08983/) (2024)
- [Adversarial Neuron Pruning Purifies Backdoored Deep Models](https://mlsanity.com/paper/2110.14430/) (2021)
- [Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection](https://mlsanity.com/paper/2307.16888/) (2023)
- [On the Exploitability of Instruction Tuning](https://mlsanity.com/paper/2306.17194/) (2023)
- [RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models](https://mlsanity.com/paper/2110.07831/) (2021)
- [BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models](https://mlsanity.com/paper/2401.12242/) (2024)
- [TrojanPuzzle: Covertly Poisoning Code-Suggestion Models](https://mlsanity.com/paper/2301.02344/) (2023)
- [Defending Pre-trained Language Models as Few-shot Learners against Backdoor Attacks](https://mlsanity.com/paper/2309.13256/) (2023)
- [QuSecNets: Quantization-based Defense Mechanism for Securing Deep Neural Network against Adversarial Attacks](https://mlsanity.com/paper/1811.01437/) (2018)
- [Exploring Backdoor Vulnerabilities of Chat Models](https://mlsanity.com/paper/2404.02406/) (2024)
- [Measuring Impacts of Poisoning on Model Parameters and Embeddings for Large Language Models of Code](https://mlsanity.com/paper/2405.11466/) (2024)

## Built on by (most cited first, 24 of 24)

- [BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models](https://mlsanity.com/paper/2408.12798/) (2024)
- [Backdoor Cleaning without External Guidance in MLLM Fine-tuning](https://mlsanity.com/paper/2505.16916/) (2025)
- [CROW: Eliminating Backdoors from Large Language Models via Internal Consistency Regularization](https://mlsanity.com/paper/2411.12768/) (2024)
- [Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Models](https://mlsanity.com/paper/2506.16447/) (2025)
- [BackdoorAgent: A Unified Framework for Backdoor Attacks on LLM-based Agents](https://mlsanity.com/paper/2601.04566/) (2026)
- [Are Robust LLM Fingerprints Adversarially Robust?](https://mlsanity.com/paper/2509.26598/) (2025)
- [AutoBackdoor: Automating Backdoor Attacks via LLM Agents](https://mlsanity.com/paper/2511.16709/) (2025)
- [Purifying Generative LLMs from Backdoors without Prior Knowledge or Clean Reference](https://mlsanity.com/paper/2603.13461/) (2026)
- [Exposing the Ghost in the Transformer: Abnormal Detection for Large Language Models via Hidden State Forensics](https://mlsanity.com/paper/2504.00446/) (2025)
- [Self-Purification Mitigates Backdoors in Multimodal Diffusion Language Models](https://mlsanity.com/paper/2602.22246/) (2026)
- [Unreal Thinking: Chain-of-Thought Hijacking via Two-stage Backdoor](https://mlsanity.com/paper/2604.09235/) (2026)
- [Lethe: Purifying Backdoored Large Language Models with Knowledge Dilution](https://mlsanity.com/paper/2508.21004/) (2025)
- [STEP: Detecting Audio Backdoor Attacks via Stability-based Trigger Exposure Profiling](https://mlsanity.com/paper/2603.18103/) (2026)
- [Layerwise Convergence Fingerprints for Runtime Misbehavior Detection in Large Language Models](https://mlsanity.com/paper/2604.24542/) (2026)
- [Inference-Time Consensus for Mitigating Hidden Behaviors from LLM Fine-Tuning](https://mlsanity.com/paper/2607.23394/) (2026)
- [Defending against Backdoor Attacks via Module Switching](https://mlsanity.com/paper/2504.05902/) (2025)
- [When Emotion Becomes Trigger: Emotion-style dynamic Backdoor Attack Parasitising Large Language Models](https://mlsanity.com/paper/2605.11612/) (2026)
- [Be Kind, Rewrite: Benign Projections via Rewriting Defend Against LLM Data Poisoning Attacks](https://mlsanity.com/paper/2605.19147/) (2026)
- [An Empirical Study of Output-to-Input Loops for Black-Box Backdoor Detection in Fine-Tuned Open-Weight LLMs](https://mlsanity.com/paper/2608.11348/) (2026)
- [SpecGuard: Inference-Time Backdoor Detection For Free](https://mlsanity.com/paper/2609.11799/) (2026)
- [Removing the NEEDLE in the Haystack: Backdoor Removal in LLMs via Weight Orthogonalisation](https://mlsanity.com/paper/2610.00348/) (2026)
- [Backdoor Containment via Expert Quarantine and Shutdown in LLMs](https://mlsanity.com/paper/2610.00663/) (2026)
- [Backdoor Purification for LoRA-Tuned LLMs via Null-Space Projection](https://mlsanity.com/paper/2610.00685/) (2026)
- [Localize-and-Detect: Auditing Task-Level Poisoning in Instruction-Tuned Models](https://mlsanity.com/paper/2610.03960/) (2026)

## Abstract (arXiv)

The remarkable performance of large language models (LLMs) in generation tasks has enabled practitioners to leverage publicly available models to power custom applications, such as chatbots and virtual assistants. However, the data used to train or fine-tune these LLMs is often undisclosed, allowing an attacker to compromise the data and inject backdoors into the models. In this paper, we develop a novel inference time defense, named CLEANGEN, to mitigate backdoor attacks for generation tasks in LLMs. CLEANGEN is a lightweight and effective decoding strategy that is compatible with the state-of-the-art (SOTA) LLMs. Our insight behind CLEANGEN is that compared to other LLMs, backdoored LLMs assign significantly higher probabilities to tokens representing the attacker-desired contents. These discrepancies in token probabilities enable CLEANGEN to identify suspicious tokens favored by the attacker and replace them with tokens generated by another LLM that is not compromised by the same attacker, thereby avoiding generation of attacker-desired content. We evaluate CLEANGEN against five SOTA backdoor attacks. Our results show that CLEANGEN achieves lower attack success rates (ASR) compared to five SOTA baseline defenses for all five backdoor attacks. Moreover, LLMs deploying CLEANGEN maintain helpfulness in their responses when serving benign user queries with minimal added computational overhead.