# Backdoor Attribution: Elucidating and Controlling Backdoor in Language Models

arXiv: https://arxiv.org/abs/2509.21761 · HTML: https://mlsanity.com/paper/2509.21761/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security.
- Builds on 41 of its 51 references that are in the map.
- First posted on arXiv 26 Sep 2025.

## Placed in

- LLM safety and security / [Backdoor and data poisoning attacks](https://mlsanity.com/safety/backdoor-and-data-poisoning-attacks/)
- Mechanistic interpretability / [Steering and probing for LLM safety](https://mlsanity.com/mechinterp/steering-and-probing-for-llm-safety/)

## Builds on (most cited first, 30 of 41)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints](https://mlsanity.com/paper/2305.13245/) (2023)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training](https://mlsanity.com/paper/2401.05566/) (2024)
- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [Function Vectors in Large Language Models](https://mlsanity.com/paper/2310.15213/) (2023)
- [Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs](https://mlsanity.com/paper/2407.15549/) (2024)
- [Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection](https://mlsanity.com/paper/2307.16888/) (2023)
- [Universal Jailbreak Backdoors from Poisoned Human Feedback](https://mlsanity.com/paper/2311.14455/) (2023)
- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [BadAgent: Inserting and Activating Backdoor Attacks in LLM Agents](https://mlsanity.com/paper/2406.03007/) (2024)
- [How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States](https://mlsanity.com/paper/2406.05644/) (2024)
- [BadEdit: Backdooring large language models by model editing](https://mlsanity.com/paper/2403.13355/) (2024)
- [A Survey on Trustworthy LLM Agents: Threats and Countermeasures](https://mlsanity.com/paper/2503.09648/) (2025)
- [Reinforcement Learning for LLM Post-Training: A Survey](https://mlsanity.com/paper/2407.16216/) (2024)
- [A Survey of LLM-based Agents in Medicine: How far are we from Baymax?](https://mlsanity.com/paper/2502.11211/) (2025)
- [BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models](https://mlsanity.com/paper/2401.12242/) (2024)
- [A Survey of Large Language Models in Finance (FinLLMs)](https://mlsanity.com/paper/2402.02315/) (2024)
- [On the Role of Attention Heads in Large Language Model Safety](https://mlsanity.com/paper/2410.13708/) (2024)
- [BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models](https://mlsanity.com/paper/2406.17092/) (2024)
- [Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons](https://mlsanity.com/paper/2406.14144/) (2024)
- [Backdoor Attacks and Countermeasures in Natural Language Processing Models: A Comprehensive Security Review](https://mlsanity.com/paper/2309.06055/) (2023)
- [Scaling Trends for Data Poisoning in LLMs](https://mlsanity.com/paper/2408.02946/) (2024)
- [A Survey on Backdoor Threats in Large Language Models (LLMs): Attacks, Defenses, and Evaluations](https://mlsanity.com/paper/2502.05224/) (2025)
- [JailbreakLens: Interpreting Jailbreak Mechanism in the Lens of Representation and Circuit](https://mlsanity.com/paper/2411.11114/) (2024)

## Built on by (most cited first, 10 of 10)

- [SafeSeek: Universal Attribution of Safety Circuits in Language Models](https://mlsanity.com/paper/2603.23268/) (2026)
- [Granulon: Awakening Pixel-Level Visual Encoders with Adaptive Multi-Granularity Semantics for MLLM](https://mlsanity.com/paper/2603.08800/) (2026)
- [Layerwise Convergence Fingerprints for Runtime Misbehavior Detection in Large Language Models](https://mlsanity.com/paper/2604.24542/) (2026)
- [Backdoor Unlearning Generalization: A Path Toward the Removal of Unknown Triggers in LLMs](https://mlsanity.com/paper/2606.03785/) (2026)
- [ProjLens: Unveiling the Role of Projectors in Multimodal Model Safety](https://mlsanity.com/paper/2604.19083/) (2026)
- [Defusing the Trigger: Tail-Risk-Informed Attention Rebalancing for LLM Backdoor Mitigation](https://mlsanity.com/paper/2604.24162/) (2026)
- [Shared Latent Structures Enable Unified Backdoor Detection and Mitigation in LLMs](https://mlsanity.com/paper/2606.07963/) (2026)
- [ReShift: Aha-Moment-Driven Reasoning-Level Backdoor Attacks on Vision-Language Models](https://mlsanity.com/paper/2607.00361/) (2026)
- [Why Are LLM Backdoor Defenses Fragmented? A Feature-Level Explanation with Sparse Autoencoders](https://mlsanity.com/paper/2608.30403/) (2026)
- [RESCUE: Repairing Language Model Errors to Sparse Circuits via Reinforcement Learning](https://mlsanity.com/paper/2609.36813/) (2026)

## Abstract (arXiv)

Fine-tuned Large Language Models (LLMs) are vulnerable to backdoor attacks through data poisoning, yet the internal mechanisms governing these attacks remain a black box. Previous research on interpretability for LLM safety tends to focus on alignment, jailbreak, and hallucination, but overlooks backdoor mechanisms, making it difficult to understand and fully eliminate the backdoor threat. In this paper, aiming to bridge this gap, we explore the interpretable mechanisms of LLM backdoors through Backdoor Attribution (BkdAttr), a tripartite causal analysis framework. We first introduce the Backdoor Probe that proves the existence of learnable backdoor features encoded within the representations. Building on this insight, we further develop Backdoor Attention Head Attribution (BAHA), efficiently pinpointing the specific attention heads responsible for processing these features. Our primary experiments reveals these heads are relatively sparse; ablating a minimal ~ 3% of total heads is sufficient to reduce the Attack Success Rate (ASR) by over 90%. More importantly, we further employ these findings to construct the Backdoor Vector derived from these attributed heads as a master controller for the backdoor. Through only 1-point intervention on single representation, the vector can either boost ASR up to ~ 100% (uparrow) on clean inputs, or completely neutralize backdoor, suppressing ASR down to ~ 0% (downarrow) on triggered inputs. In conclusion, our work pioneers the exploration of mechanistic interpretability in LLM backdoors, demonstrating a powerful method for backdoor control and revealing actionable insights for the community.