mlsanityData to 14 Oct 2026

Stealthy and Persistent Unalignment on Large Language Models via Backdoor Injections

arXiv 2312.00027 ↗First posted 15 Nov 2023cs.CR cs.AI cs.CLcited by 39

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Recent developments in Large Language Models (LLMs) have manifested significant advancements. To facilitate safeguards against malicious exploitation, a body of research has concentrated on aligning LLMs with human preferences and inhibiting their generation of inappropriate content. Unfortunately, such alignments are often vulnerable: fine-tuning with a minimal amount of harmful data can easily unalign the target LLM. While being effective, such fine-tuning-based unalignment approaches also have their own limitations: (1) non-stealthiness, after fine-tuning, safety audits or red-teaming can easily expose the potential weaknesses of the unaligned models, thereby precluding their release/use. (2) non-persistence, the unaligned LLMs can be easily repaired through re-alignment, i.e., fine-tuning again with aligned data points. In this work, we show that it is possible to conduct stealthy and persistent unalignment on large language models via backdoor injections. We also provide a novel understanding on the relationship between the backdoor persistence and the activation pattern and further provide guidelines for potential trigger design. Through extensive experiments, we demonstrate that our proposed stealthy and persistent unalignment can successfully pass the safety evaluation while maintaining strong persistence against re-alignment defense.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 29 of its 37 references that are in the map. First posted on arXiv 15 Nov 2023.

Builds on

29 of 37 references.

References · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  10. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  11. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  12. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
17 more
  1. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  2. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
  3. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  4. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  5. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 431
  6. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 336
  7. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 297
  8. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 278
  9. 2023Pretraining Language Models with Human Preferencescited by 247
  10. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 234
  11. 2019A backdoor attack against LSTM-based text classification systemscited by 227
  12. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 223
  13. 2020BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvementscited by 211
  14. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 194
  15. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 120
  16. 2023A Brief Report on LawGPT 1.0: A Virtual Legal Assistant Based on GPT-3cited by 46
  17. 2023Language Model Unalignment: Parametric Red-Teaming to Expose Hidden Harms and Biasescited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 16; 2025: 15; 2026: 8.

39 papers.

Sort:

By area:

  1. 2024Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agentscited by 137
  2. 2024Attacks, Defenses and Evaluations for LLM Conversation Safety: A Surveycited by 130
  3. 2024Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimizationcited by 118
  4. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 83
  5. 2024TrojanRAG: Retrieval-Augmented Generation Can Be Backdoor Driver in Large Language Modelscited by 67
  6. 2024BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Modelscited by 58
  7. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 46
  8. 2024A Survey of Recent Backdoor Attacks and Defenses in Large Language Modelscited by 36
  9. 2024Immunization against harmful fine-tuning attackscited by 35
  10. 2025Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Modelscited by 33
  11. 2024LoRATK: LoRA Once, Backdoor Everywhere in the Share-and-Play Ecosystemcited by 24
  12. 2024Do Influence Functions Work on Large Language Models?cited by 20
27 more
  1. 2025Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Modelscited by 19
  2. 2025BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judgecited by 18
  3. 2024Has this Fact been Edited? Detecting Knowledge Edits in Language Modelscited by 14
  4. 2024Exploring Backdoor Vulnerabilities of Chat Modelscited by 13
  5. 2025ELBA-Bench: An Efficient Learning Backdoor Attacks Benchmark for Large Language Modelscited by 11
  6. 2025Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputscited by 10
  7. 2025TrojanStego: Your Language Model Can Secretly Be A Steganographic Privacy Leaking Agentcited by 9
  8. 2025ConfGuard: A Simple and Effective Backdoor Detection for Large Language Modelscited by 9
  9. 2024Insights from the Inverse: Reconstructing LLM Training Goals Through Inverse Reinforcement Learningcited by 6
  10. 2025Backdoor Collapse: Eliminating Unknown Threats via Known Backdoor Aggregation in Language Modelscited by 6
  11. 2024Securing Multi-turn Conversational Language Models From Distributed Backdoor Triggerscited by 5
  12. 2025Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigmcited by 5
  13. 2026Character as a Latent Variable in Large Language Models: A Mechanistic Account of Emergent Misalignment and Conditional Safety Failurescited by 5
  14. 2025Where Did It Go Wrong? Attributing Undesirable LLM Behaviors via Representation Gradient Tracingcited by 4
  15. 2025Mechanistic Exploration of Backdoored Large Language Model Attention Patternscited by 3
  16. 2025Backdoor-Powered Prompt Injection Attacks Nullify Defense Methodscited by 3
  17. 2024Breaking PEFT Limitations: Leveraging Weak-to-Strong Knowledge Transfer for Backdoor Attacks in LLMscited by 2
  18. 2025BadLingual: A Novel Lingual-Backdoor Attack against Large Language Modelscited by 2
  19. 2026Structuring the Space of Sociotechnical Alignmentcited by 2
  20. 2025Self-Instruct Few-Shot Jailbreaking: Decompose the Attack into Pattern and Behavior Learningcited by 1
  21. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  22. 2026BadDLM: Backdooring Diffusion Language Models with Diverse Targetscited by 0
  23. 2026Fuzzing Large Language Models to Elicit Hidden Behaviourscited by 0
  24. 2026Anchoring Bias: A Persistent Fairness Backdoor Attack against MLLMs under Continual Learningcited by 0
  25. 2026The Safeguard Worked. Is the LLM System Safer?cited by 0
  26. 2026SilentCall: Hidden Tool-Call Backdoors in Open-Weight Agents, and How to Catch Themcited by 0
  27. 2026Towards Merging-conditional Backdoor Unalignment against Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models
  2. 2025The 'Sure' Trap: Multi-Scale Poisoning Analysis of Stealthy Compliance-Only Backdoors in Fine-Tuned Large Language Models
  3. 2024BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models
  4. 2024Jailbreaking Large Language Models Through Alignment Vulnerabilities in Out-of-Distribution Settings
  5. 2026Purifying Generative LLMs from Backdoors without Prior Knowledge or Clean Reference
  6. 2026Backdoor Unlearning Generalization: A Path Toward the Removal of Unknown Triggers in LLMs
  7. 2024MEGen: Generative Backdoor into Large Language Models via Model Editing
  8. 2026Backdoor Purification for LoRA-Tuned LLMs via Null-Space Projection