mlsanityData to 7 Oct 2026

BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning

arXiv 2108.00352 ↗First posted 1 Aug 2021cs.CR cs.CV cs.LGcited by 120

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Self-supervised learning in computer vision aims to pre-train an image encoder using a large amount of unlabeled images or (image, text) pairs. The pre-trained image encoder can then be used as a feature extractor to build downstream classifiers for many downstream tasks with a small amount of or no labeled training data. In this work, we propose BadEncoder, the first backdoor attack to self-supervised learning. In particular, our BadEncoder injects backdoors into a pre-trained image encoder such that the downstream classifiers built based on the backdoored image encoder for different downstream tasks simultaneously inherit the backdoor behavior. We formulate our BadEncoder as an optimization problem and we propose a gradient descent based method to solve it, which produces a backdoored image encoder from a clean one. Our extensive empirical evaluation results on multiple datasets show that our BadEncoder achieves high attack success rates while preserving the accuracy of the downstream classifiers. We also show the effectiveness of BadEncoder using two publicly available, real-world image encoders, i.e., Google's image encoder pre-trained on ImageNet and OpenAI's Contrastive Language-Image Pre-training (CLIP) image encoder pre-trained on 400 million (image, text) pairs collected from the Internet. Moreover, we consider defenses including Neural Cleanse and MNTD (empirical defenses) as well as PatchGuard (a provable defense). Our results show that these defenses are insufficient to defend against BadEncoder, highlighting the needs for new defenses against our BadEncoder. Our code is publicly available at: https://github.com/jjy1994/BadEncoder.

Key facts. Cited by 120 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 48 of its 69 references that are in the map. First posted on arXiv 1 Aug 2021.

Builds on

48 of 69 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  9. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  10. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  11. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  12. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
36 more
  1. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  2. 2019Strategies for Pre-training Graph Neural Networkscited by 964
  3. 2018How To Backdoor Federated Learningcited by 745
  4. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
  5. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
  6. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 461
  7. 2018Spectral Signatures in Backdoor Attackscited by 457
  8. 2019Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-trainingcited by 451
  9. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  10. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  11. 2019Hidden Trigger Backdoor Attackscited by 359
  12. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 341
  13. 2019Label-Consistent Backdoor Attackscited by 285
  14. 2019A backdoor attack against LSTM-based text classification systemscited by 214
  15. 2019Detecting AI Trojans Using Meta Neural Analysiscited by 192
  16. 2020Blind Backdoors in Deep Learning Modelscited by 174
  17. 2015Learning Visual Features from Large Weakly Supervised Datacited by 165
  18. 2021Poisoning and Backdooring Contrastive Learningcited by 149
  19. 2018SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systemscited by 148
  20. 2020Dynamic Backdoor Attacks Against Machine Learning Modelscited by 147
  21. 2018Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbationcited by 131
  22. 2020Backdoor Attacks to Graph Neural Networkscited by 126
  23. 2019Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detectioncited by 121
  24. 2019TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systemscited by 120
  25. 2020RAB: Provable Robustness Against Backdoor Attackscited by 115
  26. 2020Graph Backdoorcited by 112
  27. 2020Certified Defenses for Adversarial Patchescited by 100
  28. 2020(De)Randomized Smoothing for Certifiable Defense against Patch Attackscited by 97
  29. 2020Intrinsic Certified Robustness of Bagging against Data Poisoning Attackscited by 90
  30. 2020Trojaning Language Models for Fun and Profitcited by 80
  31. 2020On Certifying Robustness against Backdoor Attacks via Randomized Smoothingcited by 74
  32. 2019Bypassing Backdoor Detection Algorithms in Deep Learningcited by 69
  33. 2018Model-Reuse Attacks on Deep Learning Systemscited by 47
  34. 2021Efficient Certified Defenses Against Patch Attacks on Image Classifierscited by 23
  35. 2019Invisible Backdoor Attacks on Deep Neural Networks via Steganography and Regularizationcited by 17
  36. 2020PatchGuard: A Provably Robust Defense against Adversarial Patches via Small Receptive Fields and Maskingcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 27; 2023: 32; 2024: 22; 2025: 19; 2026: 15.

120 papers.

Sort:

By area:

  1. 2022Backdoor Defense via Decoupling the Training Processcited by 149
  2. 2024Optimization-based Prompt Injection Attack to LLM-as-a-Judgecited by 123
  3. 2023AdvCLIP: Downstream-agnostic Adversarial Examples in Multimodal Contrastive Learningcited by 101
  4. 2023Text-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data Poisoningcited by 91
  5. 2022BppAttack: Stealthy and Efficient Trojan Attacks against Deep Neural Networks via Image Quantization and Contrastive Adversarial Learningcited by 82
  6. 2021Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasurescited by 69
  7. 2021EncoderMI: Membership Inference against Pre-trained Encoders in Contrastive Learningcited by 68
  8. 2023VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Modelscited by 66
  9. 2022The "Beatrix'' Resurrections: Robust Backdoor Detection via Gram Matricescited by 62
  10. 2023CleanCLIP: Mitigating Data Poisoning Attacks in Multimodal Contrastive Learningcited by 62
  11. 2022Rickrolling the Artist: Injecting Backdoors into Text Encoders for Text-to-Image Synthesiscited by 59
  12. 2023Detecting Backdoors in Pre-trained Encoderscited by 53
38 more
  1. 2024Instruction Backdoor Attacks Against Customized LLMscited by 50
  2. 2023Prompt Stealing Attacks Against Text-to-Image Generation Modelscited by 48
  3. 2023UNICORN: A Unified Backdoor Trigger Inversion Frameworkcited by 44
  4. 2022SSLGuard: A Watermarking Scheme for Self-supervised Learning Pre-trained Encoderscited by 43
  5. 2022Training with More Confidence: Mitigating Injected and Natural Backdoors During Trainingcited by 38
  6. 2023Pre-trained Trojan Attacks for Visual Recognitioncited by 38
  7. 2022Rethinking the Reverse-engineering of Trojan Triggerscited by 37
  8. 2022Fine-Tuning Is All You Need to Mitigate Backdoor Attackscited by 35
  9. 2024TrojVLM: Backdoor Attack Against Vision Language Modelscited by 35
  10. 2022PoisonedEncoder: Poisoning the Unlabeled Pre-training Data in Contrastive Learningcited by 33
  11. 2023NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Modelscited by 33
  12. 2022Indiscriminate Poisoning Attacks on Unsupervised Contrastive Learningcited by 32
  13. 2023Downstream-agnostic Adversarial Examplescited by 32
  14. 2024Backdooring Vision-Language Models with Out-Of-Distribution Datacited by 31
  15. 2022Can't Steal? Cont-Steal! Contrastive Stealing Attacks Against Image Encoderscited by 30
  16. 2022An Embarrassingly Simple Backdoor Attack on Self-supervised Learningcited by 30
  17. 2022CorruptEncoder: Data Poisoning based Backdoor Attacks to Contrastive Learningcited by 28
  18. 2023Adversarial Illusions in Multi-Modal Embeddingscited by 28
  19. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 26
  20. 2023Defense-Prefix for Preventing Typographic Attacks on CLIPcited by 24
  21. 2022StolenEncoder: Stealing Pre-trained Encoders in Self-supervised Learningcited by 21
  22. 2025X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIPcited by 21
  23. 2023Backdoor Attacks Against Dataset Distillationcited by 20
  24. 2023UMD: Unsupervised Model Detection for X2X Backdoor Attackscited by 17
  25. 2023Data and Model Poisoning Backdoor Attacks on Wireless Federated Learning, and the Defense Mechanisms: A Comprehensive Surveycited by 17
  26. 2022Jigsaw Puzzle: Selective Backdoor Attack to Subvert Malware Classifierscited by 14
  27. 2022Machine Learning with Confidential Computing: A Systematization of Knowledgecited by 13
  28. 2025Seven Security Challenges in Cross-domain Multi-agent LLM Systemscited by 12
  29. 2023SSL-Cleanse: Trojan Detection and Mitigation in Self-Supervised Learningcited by 11
  30. 2022Watermarking Pre-trained Encoders in Contrastive Learningcited by 10
  31. 2024Adversarial Backdoor Defense in CLIPcited by 10
  32. 2022SSL-WM: A Black-Box Watermarking Approach for Encoders Pre-trained by Self-supervised Learningcited by 9
  33. 2022ESTAS: Effective and Stable Trojan Attacks in Self-supervised Encoders with One Target Unlabelled Samplecited by 9
  34. 2023Revisiting Data-Free Knowledge Distillation with Poisoned Teacherscited by 9
  35. 2023On the Difficulty of Defending Contrastive Learning against Backdoor Attackscited by 8
  36. 2024Mutual Information Guided Backdoor Mitigation for Pre-trained Encoderscited by 8
  37. 2022Private, Efficient, and Accurate: Protecting Models Trained by Multi-party Learning with Differential Privacycited by 7
  38. 2022Backdoor Attacks in the Supply Chain of Masked Image Modelingcited by 7

50 most cited of 120 All 120 (JSON)