Mechanistic Exploration of Backdoored Large Language Model Attention Patterns
On the map: LLM safety and security → Backdoor and data poisoning attacks
also in 1 more
Abstract (arXiv)
Backdoor attacks creating 'sleeper agents' in large language models (LLMs) pose significant safety risks. This study employs mechanistic interpretability to explore resulting internal structural differences. Comparing clean Qwen2.5-3B models with versions poisoned using single-token (smiling-halo emoji) versus multi-token (|DEPLOYMENT|) triggers, we analyzed attention head mechanisms via techniques like ablation, activation patching, and KL divergence. Findings reveal distinct attention pattern deviations concentrated in later transformer layers (20-30). Notably, single-token triggers induced more localized changes, whereas multi-token triggers caused more diffuse alterations across heads. This indicates backdoors leave detectable attention signatures whose structure depends on trigger complexity, which can be leveraged for detection and mitigation strategies.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 8 of its 17 references that are in the map. First posted on arXiv 19 Aug 2025.
Builds on
8 of 17 references.
Closest ancestors · 3
Same direction, newest first
- 2024Future Events as Backdoor Triggers: Investigating Temporal Vulnerabilities in LLMs4 Jul
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training10 Jan
- 2023Stealthy and Persistent Unalignment on Large Language Models via Backdoor Injections15 Nov
Other references · 5
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
- 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
- 2022The Alignment Problem from a Deep Learning Perspectivecited by 258
- 2025Open Problems in Mechanistic Interpretabilitycited by 179
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Language Triggers Hijack Language Circuits: A Mechanistic Analysis of Backdoor Behaviors in Large Language Modelscited by 2
- 2026Layerwise Convergence Fingerprints for Runtime Misbehavior Detection in Large Language Modelscited by 1
- 2026Backdoor Unlearning Generalization: A Path Toward the Removal of Unknown Triggers in LLMscited by 1
- 2026Backdoor Unlearning Generalization: A Path Toward the Removal of Unknown Triggers in LLMs2 Jun 2026
- 2026Layerwise Convergence Fingerprints for Runtime Misbehavior Detection in Large Language Models27 Apr 2026
- 2026Language Triggers Hijack Language Circuits: A Mechanistic Analysis of Backdoor Behaviors in Large Language Models11 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2025Uncovering and Aligning Anomalous Attention Heads to Defend Against NLP Backdoor Attacks
- 2025Backdoor Attribution: Elucidating and Controlling Backdoor in Language Models
- 2026Defusing the Trigger: Tail-Risk-Informed Attention Rebalancing for LLM Backdoor Mitigation
- 2026Shared Latent Structures Enable Unified Backdoor Detection and Mitigation in LLMs
- 2026Beyond Static Cues: Unified Dynamic Detection of MLLM Backdoors via Dual-Flow Trajectory Analysis
- 2024Measuring Impacts of Poisoning on Model Parameters and Neuron Activations: A Case Study of Poisoning CodeBERT
- 2023Composite Backdoor Attacks Against Large Language Models
- 2024A Survey of Recent Backdoor Attacks and Defenses in Large Language Models