mlsanityData to 14 Oct 2026

CleanGen: Mitigating Backdoor Attacks for Generation Tasks in Large Language Models

arXiv 2406.12257 ↗First posted 18 Jun 2024cs.AI cs.CRcited by 24

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

The remarkable performance of large language models (LLMs) in generation tasks has enabled practitioners to leverage publicly available models to power custom applications, such as chatbots and virtual assistants. However, the data used to train or fine-tune these LLMs is often undisclosed, allowing an attacker to compromise the data and inject backdoors into the models. In this paper, we develop a novel inference time defense, named CLEANGEN, to mitigate backdoor attacks for generation tasks in LLMs. CLEANGEN is a lightweight and effective decoding strategy that is compatible with the state-of-the-art (SOTA) LLMs. Our insight behind CLEANGEN is that compared to other LLMs, backdoored LLMs assign significantly higher probabilities to tokens representing the attacker-desired contents. These discrepancies in token probabilities enable CLEANGEN to identify suspicious tokens favored by the attacker and replace them with tokens generated by another LLM that is not compromised by the same attacker, thereby avoiding generation of attacker-desired content. We evaluate CLEANGEN against five SOTA backdoor attacks. Our results show that CLEANGEN achieves lower attack success rates (ASR) compared to five SOTA baseline defenses for all five backdoor attacks. Moreover, LLMs deploying CLEANGEN maintain helpfulness in their responses when serving benign user queries with minimal added computational overhead.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 29 of its 44 references that are in the map. First posted on arXiv 18 Jun 2024.

Builds on

29 of 44 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024Measuring Impacts of Poisoning on Model Parameters and Embeddings for Large Language Models of Code19 May
  2. 2024Exploring Backdoor Vulnerabilities of Chat Models3 Apr
  3. 2024BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models20 Jan
  4. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training10 Jan
  5. 2023Defending Pre-trained Language Models as Few-shot Learners against Backdoor Attacks23 Sep
  6. 2023Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection31 Jul
  7. 2023On the Exploitability of Instruction Tuning28 Jun
  8. 2021RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models15 Oct

Other references · 21

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  8. 2023Mistral 7Bcited by 3,164
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  10. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  11. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  12. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
9 more
  1. 2018Hierarchical Neural Story Generationcited by 1,193
  2. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  3. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
  4. 2023A Watermark for Large Language Modelscited by 725
  5. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 580
  6. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 247
  7. 2021Adversarial Neuron Pruning Purifies Backdoored Deep Modelscited by 235
  8. 2023TrojanPuzzle: Covertly Poisoning Code-Suggestion Modelscited by 60
  9. 2018QuSecNets: Quantization-based Defense Mechanism for Securing Deep Neural Network against Adversarial Attackscited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 7; 2026: 15.

24 papers.

Sort:

By area:

  1. 2024BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Modelscited by 50
  2. 2025Backdoor Cleaning without External Guidance in MLLM Fine-tuningcited by 35
  3. 2024CROW: Eliminating Backdoors from Large Language Models via Internal Consistency Regularizationcited by 33
  4. 2025Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Modelscited by 19
  5. 2026BackdoorAgent: A Unified Framework for Backdoor Attacks on LLM-based Agentscited by 16
  6. 2025Are Robust LLM Fingerprints Adversarially Robust?cited by 8
  7. 2025AutoBackdoor: Automating Backdoor Attacks via LLM Agentscited by 7
  8. 2026Purifying Generative LLMs from Backdoors without Prior Knowledge or Clean Referencecited by 7
  9. 2025Exposing the Ghost in the Transformer: Abnormal Detection for Large Language Models via Hidden State Forensicscited by 6
  10. 2026Self-Purification Mitigates Backdoors in Multimodal Diffusion Language Modelscited by 2
  11. 2026Unreal Thinking: Chain-of-Thought Hijacking via Two-stage Backdoorcited by 2
  12. 2025Lethe: Purifying Backdoored Large Language Models with Knowledge Dilutioncited by 1
12 more
  1. 2026STEP: Detecting Audio Backdoor Attacks via Stability-based Trigger Exposure Profilingcited by 1
  2. 2026Layerwise Convergence Fingerprints for Runtime Misbehavior Detection in Large Language Modelscited by 1
  3. 2026Inference-Time Consensus for Mitigating Hidden Behaviors from LLM Fine-Tuningcited by 1
  4. 2025Defending against Backdoor Attacks via Module Switchingcited by 0
  5. 2026When Emotion Becomes Trigger: Emotion-style dynamic Backdoor Attack Parasitising Large Language Modelscited by 0
  6. 2026Be Kind, Rewrite: Benign Projections via Rewriting Defend Against LLM Data Poisoning Attackscited by 0
  7. 2026An Empirical Study of Output-to-Input Loops for Black-Box Backdoor Detection in Fine-Tuned Open-Weight LLMscited by 0
  8. 2026SpecGuard: Inference-Time Backdoor Detection For Freecited by 0
  9. 2026Removing the NEEDLE in the Haystack: Backdoor Removal in LLMs via Weight Orthogonalisationcited by 0
  10. 2026Backdoor Containment via Expert Quarantine and Shutdown in LLMscited by 0
  11. 2026Backdoor Purification for LoRA-Tuned LLMs via Null-Space Projectioncited by 0
  12. 2026Localize-and-Detect: Auditing Task-Level Poisoning in Instruction-Tuned Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MEGen: Generative Backdoor into Large Language Models via Model Editing
  2. 2024Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models
  3. 2026BadTemplate: A Training-Free Backdoor Attack via Chat Template Against Large Language Models
  4. 2024Watch Out for Your Guidance on Generation! Exploring Conditional Backdoor Attacks against Large Language Models
  5. 2024Gracefully Filtering Backdoor Samples for Generative Large Language Models without Retraining
  6. 2023Large Language Models Are Better Adversaries: Exploring Generative Clean-Label Backdoor Attacks Against Text Classifiers
  7. 2024TrojanRAG: Retrieval-Augmented Generation Can Be Backdoor Driver in Large Language Models
  8. 2024Neutralizing Backdoors through Information Conflicts for Large Language Models