mlsanityData to 7 Oct 2026

Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch

arXiv 2106.08970 ↗First posted 16 Jun 2021cs.LG cs.CR cs.CVcited by 98

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

As the curation of data for machine learning becomes increasingly automated, dataset tampering is a mounting threat. Backdoor attackers tamper with training data to embed a vulnerability in models that are trained on that data. This vulnerability is then activated at inference time by placing a "trigger" into the model's input. Typical backdoor attacks insert the trigger directly into the training data, although the presence of such an attack may be visible upon inspection. In contrast, the Hidden Trigger Backdoor Attack achieves poisoning without placing a trigger into the training data at all. However, this hidden trigger attack is ineffective at poisoning neural networks trained from scratch. We develop a new hidden trigger attack, Sleeper Agent, which employs gradient matching, data selection, and target model re-training during the crafting process. Sleeper Agent is the first hidden trigger backdoor attack to be effective against neural networks trained from scratch. We demonstrate its effectiveness on ImageNet and in black-box settings. Our implementation code can be found at https://github.com/hsouri/Sleeper-Agent.

Key facts. Cited by 98 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 35 of its 63 references that are in the map. First posted on arXiv 16 Jun 2021.

Builds on

35 of 63 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  6. 2016Deep Learning with Differential Privacycited by 2,682
  7. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  8. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  9. 2018How To Backdoor Federated Learningcited by 745
  10. 2020Dataset Condensation with Gradient Matchingcited by 510
  11. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
  12. 2018Spectral Signatures in Backdoor Attackscited by 457
23 more
  1. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  2. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  3. 2020Backdoor Learning: A Surveycited by 412
  4. 2019Hidden Trigger Backdoor Attackscited by 359
  5. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 341
  6. 2020Invisible Backdoor Attack with Sample-Specific Triggerscited by 338
  7. 2017Certified Defenses for Data Poisoning Attackscited by 288
  8. 2019Label-Consistent Backdoor Attackscited by 285
  9. 2019A new Backdoor Attack in CNNs by training set corruption without label poisoningcited by 237
  10. 2017Embedding Watermarks into Deep Neural Networkscited by 227
  11. 2017Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimizationcited by 188
  12. 2021Unlearnable Examples: Making Personal Data Unexploitablecited by 188
  13. 2020Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defensescited by 171
  14. 2020Witches' Brew: Industrial Scale Data Poisoning via Gradient Matchingcited by 156
  15. 2019Transferable Clean-Label Poisoning Attacks on Deep Neural Netscited by 129
  16. 2020Backdoor Attacks Against Deep Learning Systems in the Physical Worldcited by 129
  17. 2020MetaPoison: Practical General-purpose Clean-label Data Poisoningcited by 114
  18. 2020Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attackscited by 109
  19. 2020On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shapingcited by 82
  20. 2019Learning to Confuse: Generating Training Time Adversarial Data with Auto-Encodercited by 70
  21. 2021Preventing Unauthorized Use of Proprietary Data: Poisoning for Secure Dataset Releasecited by 35
  22. 2021DP-InstaHide: Provably Defusing Poisoning and Backdoor Attacks with Differentially Private Data Augmentationscited by 33
  23. 2019Invisible Backdoor Attacks on Deep Neural Networks via Steganography and Regularizationcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 13; 2023: 26; 2024: 28; 2025: 15; 2026: 14.

98 papers.

Sort:

By area:

  1. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
  2. 2022BackdoorBench: A Comprehensive Benchmark of Backdoor Learningcited by 124
  3. 2021Adversarial Examples Make Strong Poisonscited by 115
  4. 2022Narcissus: A Practical Clean-Label Backdoor Attack with Limited Informationcited by 102
  5. 2022Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoningcited by 93
  6. 2022Untargeted Backdoor Watermark: Towards Harmless and Stealthy Dataset Copyright Protectioncited by 80
  7. 2023Domain Watermark: Effective and Harmless Dataset Copyright Protection is Closed at Handcited by 52
  8. 2023Detecting Backdoors During the Inference Stage Based on Corruption Robustness Consistencycited by 45
  9. 2023Black-box Backdoor Defense via Zero-shot Image Purificationcited by 44
  10. 2023Shared Adversarial Unlearning: Backdoor Mitigation by Unlearning Shared Adversarial Examplescited by 40
  11. 2023Attention-Enhancing Backdoor Attacks Against BERT-based Modelscited by 32
  12. 2024Mitigating Fine-tuning based Jailbreak Attack with Backdoor Enhanced Safety Alignmentcited by 31
38 more
  1. 2022Not All Poisons are Created Equal: Robust Training against Data Poisoningcited by 29
  2. 2023ASSET: Robust Backdoor Data Detection Across a Multiplicity of Deep Learning Paradigmscited by 29
  3. 2023Towards Stealthy Backdoor Attacks against Speech Recognition via Elements of Soundcited by 28
  4. 2023BackdoorBox: A Python Toolbox for Backdoor Learningcited by 25
  5. 2023Stable Unlearnable Example: Enhancing the Robustness of Unlearnable Examples via Stable Error-Minimizing Noisecited by 23
  6. 2022Hardly Perceptible Trojan Attack against Neural Networks with Bit Flipscited by 21
  7. 2022Friendly Noise against Adversarial Noise: A Powerful Defense against Data Poisoning Attackscited by 20
  8. 2023Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspectivecited by 19
  9. 2022Plug-In Inversion: Model-Agnostic Inversion for Vision with Data Augmentationscited by 18
  10. 2024BackdoorBench: A Comprehensive Benchmark and Analysis of Backdoor Learningcited by 17
  11. 2025Natural Reflection Backdoor Attack on Vision Language Model for Autonomous Drivingcited by 16
  12. 2022On the Effectiveness of Adversarial Training against Backdoor Attackscited by 15
  13. 2024Generalization Bound and New Algorithm for Clean-Label Backdoor Attackcited by 11
  14. 2022Imperceptible and Multi-channel Backdoor Attack against Deep Neural Networkscited by 10
  15. 2023Temporal Robustness against Data Poisoningcited by 10
  16. 2023Demystifying Poisoning Backdoor Attacks from a Statistical Perspectivecited by 10
  17. 2023Rethinking Backdoor Attacks on Dataset Distillation: A Kernel Method Perspectivecited by 10
  18. 2024Mitigating Backdoor Attack by Injecting Proactive Defensive Backdoorcited by 10
  19. 2023Revisiting Data-Free Knowledge Distillation with Poisoned Teacherscited by 9
  20. 2024UIBDiffusion: Universal Imperceptible Backdoor Attack for Diffusion Modelscited by 9
  21. 2024Data Taggants: Dataset Ownership Verification via Harmless Targeted Data Poisoningcited by 8
  22. 2023Efficient Backdoor Attacks for Deep Neural Networks in Real-world Scenarioscited by 7
  23. 2025Hidden in the Noise: Unveiling Backdoors in Audio LLMs Alignment through Latent Acoustic Pattern Triggerscited by 7
  24. 2023Adversarial Backdoor Attack by Naturalistic Data Poisoning on Trajectory Prediction in Autonomous Drivingcited by 5
  25. 2023Boosting Backdoor Attack with A Learnable Poisoning Sample Selection Strategycited by 5
  26. 2023FilterFL: Knowledge Filtering-based Data-Free Backdoor Defense for Federated Learningcited by 5
  27. 2024Test-Time Multimodal Backdoor Detection by Contrastive Promptingcited by 5
  28. 2024IPA-NeRF: Illusory Poisoning Attack Against Neural Radiance Fieldscited by 5
  29. 2023Prompt Backdoors in Visual Prompt Learningcited by 4
  30. 2024TrojFM: Resource-efficient Backdoor Attacks against Very Large Foundation Modelscited by 4
  31. 2024Clean-Label Physical Backdoor Attacks with Data Distillationcited by 4
  32. 2024Anonymity Unveiled: A Practical Framework for Auditing Data Use in Deep Learning Modelscited by 4
  33. 2024PureGen: Universal Data Purification for Train-Time Poison Defense via Generative Model Dynamicscited by 3
  34. 2024UTrace: Poisoning Forensics for Private Collaborative Learningcited by 3
  35. 2024BackdoorMBTI: A Backdoor Learning Multimodal Benchmark Tool Kit for Backdoor Defense Evaluationcited by 3
  36. 2025Improving the Sensitivity of Backdoor Detectors via Class Subspace Orthogonalizationcited by 3
  37. 2026CBV: Clean-label Backdoor Attacks on Vision Language Models via Diffusion Modelscited by 3
  38. 2022Under-confidence Backdoors Are Resilient and Stealthy Backdoorscited by 2

50 most cited of 98 All 98 (JSON)