# Pay Attention to the Triggers: Constructing Backdoors That Survive Distillation

arXiv: https://arxiv.org/abs/2510.18541 · HTML: https://mlsanity.com/paper/2510.18541/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Federated, robust and private distillation, one of 12 directions in Knowledge distillation.
- Builds on 36 of its 46 references that are in the map.
- First posted on arXiv 21 Oct 2025.

## Placed in

- Knowledge distillation / [Federated, robust and private distillation](https://mlsanity.com/distill/federated-robust-and-private-distillation/)

## Builds on (most cited first, 30 of 36)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Weight Poisoning Attacks on Pre-trained Models](https://mlsanity.com/paper/2004.06660/) (2020)
- [Poisoning Language Models During Instruction Tuning](https://mlsanity.com/paper/2305.00944/) (2023)
- [A backdoor attack against LSTM-based text classification systems](https://mlsanity.com/paper/1905.12457/) (2019)
- [Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger](https://mlsanity.com/paper/2105.12400/) (2021)
- [BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvements](https://mlsanity.com/paper/2006.01043/) (2020)
- [OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data](https://mlsanity.com/paper/2410.01560/) (2024)
- [Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs](https://mlsanity.com/paper/2407.15549/) (2024)
- [Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection](https://mlsanity.com/paper/2307.16888/) (2023)
- [Universal Jailbreak Backdoors from Poisoned Human Feedback](https://mlsanity.com/paper/2311.14455/) (2023)
- [On the Exploitability of Instruction Tuning](https://mlsanity.com/paper/2306.17194/) (2023)
- [Subliminal Learning: Language models transmit behavioral traits via hidden signals in data](https://mlsanity.com/paper/2507.14805/) (2025)
- [Composite Backdoor Attacks Against Large Language Models](https://mlsanity.com/paper/2310.07676/) (2023)
- [Instruction Backdoor Attacks Against Customized LLMs](https://mlsanity.com/paper/2402.09179/) (2024)
- [Exploring the Universal Vulnerability of Prompt-based Learning Paradigm](https://mlsanity.com/paper/2204.05239/) (2022)
- [BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models](https://mlsanity.com/paper/2408.12798/) (2024)
- [Persistent Pre-Training Poisoning of LLMs](https://mlsanity.com/paper/2410.13722/) (2024)

## Built on by (most cited first, 6 of 6)

- [From Parameters to Behaviors: A Survey of Model Fusion for Large Language Models](https://mlsanity.com/paper/2609.19553/) (2026)
- [Widening the Gap: Exploiting LLM Quantization via Outlier Injection](https://mlsanity.com/paper/2605.15152/) (2026)
- [No Free Efficiency: Revisiting the Trade-off Between Training Efficiency and Model Vulnerability](https://mlsanity.com/paper/2609.33898/) (2026)
- [Self-Propagating Misalignment in LLM Agents, and Why Auditing or Disabling Memory Is Not Enough](https://mlsanity.com/paper/2610.04083/) (2026)
- [Does On-Policy Distillation for Safety Pose Backdoor Risks?](https://mlsanity.com/paper/2610.07654/) (2026)
- [Imprinting: Knowledge Distillation Can Shift Student Behavior](https://mlsanity.com/paper/2610.18517/) (2026)

## Abstract (arXiv)

LLMs are often used by downstream users as teacher models for knowledge distillation, compressing their capabilities into memory-efficient models. However, as these teacher models may stem from untrusted parties, distillation can raise unexpected security risks. In this paper, we investigate the security implications of knowledge distillation from backdoored teacher models. First, we show that prior backdoors mostly do not transfer onto student models. Our key insight is that this is because existing LLM backdooring methods choose trigger tokens that rarely occur in usual contexts. We argue that this underestimates the security risks of knowledge distillation and introduce a new backdooring technique, T-MTB, that enables the construction and study of transferable backdoors. T-MTB carefully constructs a composite backdoor trigger, made up of several specific tokens that often occur individually in anticipated distillation datasets. As such, the poisoned teacher remains stealthy, while during distillation the individual presence of these tokens provides enough signal for the backdoor to transfer onto the student. Using T-MTB, we demonstrate and extensively study the security risks of transferable backdoors across two attack scenarios, jailbreaking and content modulation, and across four model families of LLMs.