mlsanityData to 7 Oct 2026

Backdoor Collapse: Eliminating Unknown Threats via Known Backdoor Aggregation in Language Models

arXiv 2510.10265 ↗First posted 11 Oct 2025cs.CLcited by 5

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Backdoor attacks are a significant threat to large language models (LLMs), often embedded via public checkpoints, yet existing defenses rely on impractical assumptions about trigger settings. To address this challenge, we propose ourmethod, a defense framework that requires no prior knowledge of trigger settings. ourmethod is based on the key observation that when deliberately injecting known backdoors into an already-compromised model, both existing unknown and newly injected backdoors aggregate in the representation space. ourmethod leverages this through a two-stage process: first, aggregating backdoor representations by injecting known triggers, and then, performing recovery fine-tuning to restore benign outputs. Extensive experiments across multiple LLM architectures demonstrate that: (I) ourmethod reduces the average Attack Success Rate to 4.41% across multiple benchmarks, outperforming existing baselines by 28.1%~69.3%uparrow. (II) Clean accuracy and utility are preserved within 0.5% of the original model, ensuring negligible impact on legitimate tasks. (III) The defense generalizes across different types of backdoors, confirming its robustness in practical deployment scenarios.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 26 of its 38 references that are in the map. First posted on arXiv 11 Oct 2025.

Builds on

26 of 38 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoning11 Oct
  2. 2024Scaling Trends for Data Poisoning in LLMs6 Aug
  3. 2024BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models24 Jun
  4. 2024BadEdit: Backdooring large language models by model editing20 Mar
  5. 2023Stealthy and Persistent Unalignment on Large Language Models via Backdoor Injections15 Nov
  6. 2022Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models18 Oct
  7. 2021What Doesn't Kill You Makes You Robust(er): How to Adversarially Train against Data Poisoning26 Feb

Other references · 19

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  2. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  3. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  4. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  5. 2023Detecting Language Model Attacks with Perplexitycited by 411
  6. 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
  7. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 231
  8. 2024Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agentscited by 135
  9. 2023Universal Jailbreak Backdoors from Poisoned Human Feedbackcited by 120
  10. 2024BadAgent: Inserting and Activating Backdoor Attacks in LLM Agentscited by 106
  11. 2022Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLPcited by 99
  12. 2024Defending Against Unforeseen Failure Modes with Latent Adversarial Trainingcited by 82
7 more
  1. 2024Best-of-Venom: Attacking RLHF by Injecting Poisoned Preference Datacited by 33
  2. 2022Trap and Replace: Defending Backdoor Attacks by Trapping Them into an Easy-to-Replace Subnetworkcited by 12
  3. 2024Mitigating Backdoor Attack by Injecting Proactive Defensive Backdoorcited by 10
  4. 2024Modeling Spatio-temporal Dynamical Systems with Neural Discrete Learning and Levels-of-Expertscited by 9
  5. 2025SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Modelscited by 8
  6. 2025Hidden in the Noise: Unveiling Backdoors in Audio LLMs Alignment through Latent Acoustic Pattern Triggerscited by 7
  7. 2025ERIS: Evolutionary Real-world Interference Scheme for Jailbreaking Audio Large Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026SafeSeek: Universal Attribution of Safety Circuits in Language Modelscited by 4
  2. 2026ProjLens: Unveiling the Role of Projectors in Multimodal Model Safetycited by 0
  3. 2026Shared Latent Structures Enable Unified Backdoor Detection and Mitigation in LLMscited by 0
  4. 2026Backdoor Decontamination Dynamics in LLM Agentscited by 0
  5. 2026Backdoor in the Loop: Compromising Agentic Search via Malicious Retrieverscited by 0