mlsanityData to 7 Oct 2026

BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models

arXiv 2406.17092 ↗First posted 24 Jun 2024cs.CR cs.AIcited by 50

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Safety backdoor attacks in large language models (LLMs) enable the stealthy triggering of unsafe behaviors while evading detection during normal interactions. The high dimensionality of potential triggers in the token space and the diverse range of malicious behaviors make this a critical challenge. We present BEEAR, a mitigation approach leveraging the insight that backdoor triggers induce relatively uniform drifts in the model's embedding space. Our bi-level optimization method identifies universal embedding perturbations that elicit unwanted behaviors and adjusts the model parameters to reinforce safe behaviors against these perturbations. Experiments show BEEAR reduces the success rate of RLHF time backdoor attacks from >95% to <1% and from 47% to 0% for instruction-tuning time backdoors targeting malicious code generation, without compromising model utility. Requiring only defender-defined safe and unwanted behaviors, BEEAR represents a step towards practical defenses against safety backdoors in LLMs, providing a foundation for further advancements in AI safety and security.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 45 of its 69 references that are in the map. First posted on arXiv 24 Jun 2024.

Builds on

45 of 69 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training10 Jan
  2. 2023Stealthy and Persistent Unalignment on Large Language Models via Backdoor Injections15 Nov
  3. 2023BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPT21 Feb
  4. 2022Rethinking the Reverse-engineering of Trojan Triggers27 Oct
  5. 2022Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models18 Oct
  6. 2021Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasures9 Dec
  7. 2021RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models15 Oct
  8. 2021T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text Classification7 Mar

Other references · 37

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2017Deep reinforcement learning from human preferencescited by 3,846
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  5. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  6. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  7. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  8. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  9. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  10. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
  11. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  12. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
25 more
  1. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  2. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  3. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
  4. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  5. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 417
  6. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  7. 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
  8. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
  9. 2023Poisoning Language Models During Instruction Tuningcited by 254
  10. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
  11. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 231
  12. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 183
  13. 2021Adversarial Unlearning of Backdoors via Implicit Hypergradientcited by 160
  14. 2019TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systemscited by 120
  15. 2023Universal Jailbreak Backdoors from Poisoned Human Feedbackcited by 120
  16. 2021AdaSpeech: Adaptive Text to Speech for Custom Voicecited by 98
  17. 2023LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representationscited by 81
  18. 2020Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identificationcited by 80
  19. 2019Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networkscited by 66
  20. 2024Emulated Disalignment: Safety Alignment for Large Language Models May Backfire!cited by 36
  21. 2023BadLlama: cheaply removing safety fine-tuning from Llama 2-Chat 13Bcited by 35
  22. 2022Post-Training Detection of Backdoor Attacks for Two-Class and Multi-Attack Scenarioscited by 29
  23. 2024Competition Report: Finding Universal Jailbreak Backdoors in Aligned LLMscited by 27
  24. 2023From Shortcuts to Triggers: Backdoor Defense with Denoised PoEcited by 26
  25. 2024What Should Embeddings Embed? Autoregressive Models Represent Latent Generating Distributionscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 22; 2026: 18.

50 papers.

Sort:

By area:

  1. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  2. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 86
  3. 2024Persistent Pre-Training Poisoning of LLMscited by 46
  4. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  5. 2024BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Modelscited by 44
  6. 2024NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuningcited by 37
  7. 2025Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiescited by 36
  8. 2024Do Influence Functions Work on Large Language Models?cited by 20
  9. 2025Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasetscited by 18
  10. 2025Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Modelscited by 15
  11. 2024Mitigating the Backdoor Effect for Multi-Task Model Merging via Safety-Aware Subspacecited by 14
  12. 2026The Trigger in the Haystack: Extracting and Reconstructing LLM Backdoor Triggerscited by 13
38 more
  1. 2025Scalable Fingerprinting of Large Language Modelscited by 11
  2. 2025Life-Cycle Routing Vulnerabilities of LLM Routercited by 10
  3. 2025ICLShield: Exploring and Mitigating In-Context Learning Backdoor Attackscited by 9
  4. 2025Efficient Input-level Backdoor Defense on Text-to-Image Synthesis via Neuron Activation Variationcited by 8
  5. 2025Backdoor Attribution: Elucidating and Controlling Backdoor in Language Modelscited by 8
  6. 2025Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMscited by 7
  7. 2025BadReasoner: Planting Tunable Overthinking Backdoors into Large Reasoning Models for Fun or Profitcited by 6
  8. 2025Same Question, Different Words: A Latent Adversarial Framework for Prompt Robustnesscited by 5
  9. 2025Backdoor Collapse: Eliminating Unknown Threats via Known Backdoor Aggregation in Language Modelscited by 5
  10. 2024Neutralizing Backdoors through Information Conflicts for Large Language Modelscited by 4
  11. 2025Spectral Insights into Data-Oblivious Critical Layers in Large Language Modelscited by 4
  12. 2025Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigmcited by 4
  13. 2026SafeSeek: Universal Attribution of Safety Circuits in Language Modelscited by 4
  14. 2026Why LoRA Fails to Forget: Regularized Low-Rank Adaptation Against Backdoors in Language Modelscited by 3
  15. 2025NLP Security and Ethics, in the Wildcited by 2
  16. 2025Dual-Space Smoothness for Robust and Balanced LLM Unlearningcited by 2
  17. 2025From Poisoned to Aware: Fostering Backdoor Self-Awareness in LLMscited by 2
  18. 2025Rethinking Reasoning: A Survey on Reasoning-based Backdoors in LLMscited by 1
  19. 2025GREAT: Generalizable Backdoor Attacks in RLHF via Emotion-Aware Trigger Synthesiscited by 1
  20. 2025SCOUT: A Defense Against Data Poisoning Attacks in Fine-Tuned Language Modelscited by 1
  21. 2026TrojanPraise: Jailbreak LLMs via Benign Fine-Tuningcited by 1
  22. 2026Backdooring Bias in Large Language Modelscited by 1
  23. 2026Security in the Fine-Tuning Lifecycle of Large Language Models: Threats, Defenses,Evaluation, and Future Directionscited by 1
  24. 2026Poison with Style: A Practical Poisoning Attack on Code Large Language Modelscited by 1
  25. 2026Dummy Backdoor as a Defense: Removing Unknown Backdoors via Shared Internal Mechanisms for Generative LLMscited by 1
  26. 2024BiasJailbreak:Analyzing Ethical Biases and Jailbreak Vulnerabilities in Large Language Modelscited by 0
  27. 2025Lethe: Purifying Backdoored Large Language Models with Knowledge Dilutioncited by 0
  28. 2025Inverting Trojans in LLMscited by 0
  29. 2026STEP: Detecting Audio Backdoor Attacks via Stability-based Trigger Exposure Profilingcited by 0
  30. 2026Defusing the Trigger: Tail-Risk-Informed Attention Rebalancing for LLM Backdoor Mitigationcited by 0
  31. 2026Most Current Model Organisms Are Leaky: Perplexity Differencing Often Reveals Finetuning Objectivescited by 0
  32. 2026BadDLM: Backdooring Diffusion Language Models with Diverse Targetscited by 0
  33. 2026Fuzzing Large Language Models to Elicit Hidden Behaviourscited by 0
  34. 2026CSO-LLM: Class Subspace Orthogonalization for Post-Training Backdoor Detection and Trigger Inversion in LLMscited by 0
  35. 2026ToxScreen: Detecting Whether an LLM Has Been Poisonedcited by 0
  36. 2026Detecting Hidden Behaviors in LLMs via Activation-matched Finetuningcited by 0
  37. 2026SilentCall: Hidden Tool-Call Backdoors in Open-Weight Agents, and How to Catch Themcited by 0
  38. 2026Removing the NEEDLE in the Haystack: Backdoor Removal in LLMs via Weight Orthogonalisationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Patcher: Post-Hoc Patching of Backdoored Large Language Models
  2. 2025Robust Anti-Backdoor Instruction Tuning in LVLMs
  3. 2024Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models
  4. 2025AutoBackdoor: Automating Backdoor Attacks via LLM Agents
  5. 2024Mitigating Backdoor Threats to Large Language Models: Advancement and Challenges
  6. 2025DualEdit: Mitigating Safety Fallback in LLM Backdoor Editing via Affirmation-Refusal Regulation
  7. 2024Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift
  8. 2026Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors