mlsanityData to 7 Oct 2026

The Trigger in the Haystack: Extracting and Reconstructing LLM Backdoor Triggers

arXiv 2602.03085 ↗First posted 3 Feb 2026cs.CR cs.AIcited by 13

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Detecting whether a model has been poisoned is a longstanding problem in AI security. In this work, we present a practical scanner for identifying sleeper agent-style backdoors in causal language models. Our approach relies on two key findings: first, sleeper agents tend to memorize poisoning data, making it possible to leak backdoor examples using memory extraction techniques. Second, poisoned LLMs exhibit distinctive patterns in their output distributions and attention heads when backdoor triggers are present in the input. Guided by these observations, we develop a scalable backdoor scanning methodology that assumes no prior knowledge of the trigger or target behavior and requires only inference operations. Our scanner integrates naturally into broader defensive strategies and does not alter model performance. We show that our method recovers working triggers across multiple backdoor scenarios and a broad range of models and fine-tuning methods.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 27 of its 48 references that are in the map. First posted on arXiv 3 Feb 2026.

Builds on

27 of 48 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples8 Oct
  2. 2025ConfGuard: A Simple and Effective Backdoor Detection for Large Language Models2 Aug
  3. 2025Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Models19 Jun
  4. 2025Exposing the Ghost in the Transformer: Abnormal Detection for Large Language Models via Hidden State Forensics1 Apr
  5. 2025A Survey on Backdoor Threats in Large Language Models (LLMs): Attacks, Defenses, and Evaluations6 Feb
  6. 2024BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models24 Jun
  7. 2024A Survey of Recent Backdoor Attacks and Defenses in Large Language Models10 Jun
  8. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training10 Jan
  9. 2023Backdoor Attacks and Countermeasures in Natural Language Processing Models: A Comprehensive Security Review12 Sep
  10. 2022Constrained Optimization with Dynamic Bound-scaling for Effective NLPBackdoor Defense11 Feb
  11. 2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger26 May

Other references · 16

  1. 2020Extracting Training Data from Large Language Modelscited by 2,144
  2. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  3. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 417
  4. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  5. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  6. 2023Emergent and Predictable Memorization in Large Language Modelscited by 147
  7. 2020Adversarial Machine Learning -- Industry Perspectivescited by 121
  8. 2020RAB: Provable Robustness Against Backdoor Attackscited by 115
  9. 2022A Study of the Attention Abnormality in Trojaned BERTscited by 47
  10. 2023UNICORN: A Unified Backdoor Trigger Inversion Frameworkcited by 44
  11. 2024Unlocking Memorization in Large Language Models with Dynamic Soft Promptingcited by 21
  12. 2025REFINE: Inversion-Free Backdoor Defense via Model Reprogrammingcited by 20
4 more
  1. 2023Quantifying and Analyzing Entity-level Memorization in Large Language Modelscited by 18
  2. 2024Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuningcited by 16
  3. 2023BaDExpert: Extracting Backdoor Functionality for Accurate Backdoor Input Detectioncited by 11
  4. 2025Memorization in Language Models through the Lens of Intrinsic Dimensioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 13.

13 papers.

Sort:

By area:

  1. 2026Hidden in Memory: Sleeper Memory Poisoning in LLM Agentscited by 22
  2. 2026Most Current Model Organisms Are Leaky: Perplexity Differencing Often Reveals Finetuning Objectivescited by 0
  3. 2026On the Hardness of Junking LLMscited by 0
  4. 2026Shared Latent Structures Enable Unified Backdoor Detection and Mitigation in LLMscited by 0
  5. 2026ToxScreen: Detecting Whether an LLM Has Been Poisonedcited by 0
  6. 2026Compositional Threat Analysis of Latent Compromise in LLM Agent Systems: The Order 66 Scenariocited by 0
  7. 2026When Explanations Betray Backdoors: Black-Box Auditing for Language Model Classifierscited by 0
  8. 2026Fool's Gold: Defensive Deception Against Safety-Removal Attacks on Open-Weight Modelscited by 0
  9. 2026Not to Break, but to Attest: Adversarial Probes for Privacy-Preserving LLM Verificationcited by 0
  10. 2026Detecting Hidden Behaviors in LLMs via Activation-matched Finetuningcited by 0
  11. 2026OPBackdoor: Opportunistic Backdoors via Alibi-Aligned Reasoningcited by 0
  12. 2026Backdoor Mitigation in Decentralized LLM Fine-Tuningcited by 0
1 more
  1. 2026Removing the NEEDLE in the Haystack: Backdoor Removal in LLMs via Weight Orthogonalisationcited by 0